Multi-source heterogeneous industrial data decision-making method and system based on model matrix

Abstract

The invention provides a multi-source heterogeneous industrial data decision-making method and system based on a model matrix in the technical field of industrial data processing and intelligent decision-making. The method comprises the steps of S1, collecting multi-source industrial data; s2, preprocessing each piece of multi-source industrial data to construct a multi-modal data matrix; s3, constructing a model matrix based on the model layer and the agent layer, and setting an API interface for calling the model layer; s4, acquiring an input business demand, inputting the business demand to the model matrix, and matching a corresponding agent from the agent layer by the model matrix based on the demand type of the business demand; and S5, the intelligent agent matches corresponding industrial data from the multi-modal data matrix based on the service demand, and performs collaborative reasoning on the industrial data based on an adaptive AI model in the API interface scheduling model layer to obtain a decision suggestion corresponding to the service demand. The method has the advantages that the accuracy of industrial data decision making and the intelligent level of the system are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of industrial data processing and intelligent decision-making technology, and in particular to a multi-source heterogeneous industrial data decision-making method and system based on model matrices. Background Technology

[0002] With the deepening advancement of Industry 4.0, industrial data is experiencing explosive growth. This data encompasses both structured sensor data such as temperature, pressure, and rotational speed, and unstructured information such as a large amount of process manuals, technical documents, design drawings, and production site images. Achieving efficient integration and intelligent analysis of this multi-source, heterogeneous data has become a key link in promoting the intelligent development of industry, directly impacting the accuracy of industrial decision-making and the intelligence level of production systems.

[0003] Currently, the industrial sector primarily employs the following practices in data processing and model building: At the data integration level, it relies heavily on traditional structured databases and data warehouse technologies, while unstructured data is often stored independently or indexed using simple methods. This lack of a unified semantic association and deep fusion mechanism leads to significant information silos. At the model building level, it typically uses dedicated models for single data types or simply strings multiple models together at the task level, making it difficult to achieve cross-modal semantic alignment and joint reasoning capabilities. Taking equipment fault diagnosis as an example, existing methods often rely solely on sensor values ​​for threshold judgments or simple time-series analysis, failing to effectively integrate unstructured information such as equipment history records, maintenance reports, and on-site images. This results in an incomplete diagnostic process and limited traceability. In the product design phase, design drawings and technical documents are often stored separately, with visual information and textual knowledge fragmented, hindering the integrated process and efficiency improvement of design optimization.

[0004] In summary, the existing technology mainly has the following problems: 1. Difficulty in data integration: Traditional data processing methods struggle to achieve semantic-level fusion of multi-source heterogeneous data. Structured and unstructured data are independent of each other, with weak information correlation, making it impossible to build a unified data view and limiting the realization of the comprehensive value of data.

[0005] 2. Poor model adaptability: Faced with complex and ever-changing industrial scenarios, existing models are mostly single-modal or simple combinations, lacking the ability to jointly semantically model cross-modal data such as text and images. This results in large semantic understanding biases and limited prediction accuracy, making it difficult to support the intelligent analysis needs of complex industrial tasks.

[0006] 3. Limitations in Application Scenarios: In specific scenarios such as fault diagnosis and product design, existing methods fail to achieve effective collaboration of multi-source information. For example, unstructured information is difficult to integrate into the analysis process during fault diagnosis, resulting in low diagnostic efficiency and a high misjudgment rate; in product design, the separation of text and graphics information affects the comprehensiveness of design optimization, prolongs the R&D cycle, and increases development costs.

[0007] Therefore, how to provide a multi-source heterogeneous industrial data decision-making method and system based on model matrices to improve the accuracy of industrial data decision-making and the level of system intelligence has become an urgent technical problem to be solved. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a multi-source heterogeneous industrial data decision-making method and system based on model matrix, so as to improve the accuracy of industrial data decision-making and the level of system intelligence.

[0009] In a first aspect, the present invention provides a multi-source heterogeneous industrial data decision-making method based on a model matrix, comprising the following steps: Step S1: Collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data; Step S2: Preprocess the multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix; Step S3: Construct a model matrix based on the model layer and the agent layer, and set the API interface for calling the model layer; Step S4: Obtain the input business requirements and input the business requirements into the model matrix. The model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirements. Step S5: The intelligent agent matches corresponding industrial data from the multimodal data matrix based on business requirements, performs collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtains decision suggestions corresponding to the business requirements, and outputs the decision suggestions.

[0010] Furthermore, step S1 specifically includes: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

[0011] Furthermore, step S2 specifically includes: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation. The preprocessed industrial structured data is then stored in a time-series database with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition. It identifies knowledge elements, including entities, attributes, and relationships, from the preprocessed unstructured data of each document. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, it constructs a knowledge graph and maps the unstructured data of each document to the knowledge graph to construct a knowledge graph data matrix. The server performs at least local feature extraction, downsampling, and feature integration on the unstructured data of each image to obtain a fixed-length image feature vector. It also labels the unstructured data of each image with at least category, key design elements, and product information to complete the preprocessing of the unstructured data of each image. Based on the unstructured data of each image, the image feature vector, and the labeled tags, an image data matrix is ​​constructed. A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

[0012] Furthermore, step S3 specifically includes: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

[0013] Furthermore, step S5 specifically includes: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models in the model layer based on the API interface, dynamically allocates computing resources for each AI model, and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision recommendations in a structured manner in natural language.

[0014] Secondly, this invention provides a multi-source heterogeneous industrial data decision-making system based on a model matrix, comprising the following modules: The multi-source industrial data acquisition module is used to collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data. A multimodal data matrix construction module is used to preprocess the various multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix. The model matrix construction module is used to construct a model matrix based on the model layer and the agent layer, and to set the API interface for calling the model layer. The business requirement input module is used to obtain input business requirements and input the business requirements into the model matrix. The model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirements. The decision suggestion output module is used by the intelligent agent to match corresponding industrial data from the multimodal data matrix based on business needs, perform collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtain decision suggestions corresponding to the business needs, and output the decision suggestions.

[0015] Furthermore, the multi-source industrial data acquisition module is specifically used for: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

[0016] Furthermore, the multimodal data matrix construction module is specifically used for: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation. The preprocessed industrial structured data is then stored in a time-series database with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition. It identifies knowledge elements, including entities, attributes, and relationships, from the preprocessed unstructured data of each document. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, it constructs a knowledge graph and maps the unstructured data of each document to the knowledge graph to construct a knowledge graph data matrix. The server performs at least local feature extraction, downsampling, and feature integration on the unstructured data of each image to obtain a fixed-length image feature vector. It also labels the unstructured data of each image with at least category, key design elements, and product information to complete the preprocessing of the unstructured data of each image. Based on the unstructured data of each image, the image feature vector, and the labeled tags, an image data matrix is ​​constructed. A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

[0017] Furthermore, the model matrix construction module is specifically used for: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

[0018] Furthermore, the decision suggestion output module is specifically used for: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models in the model layer based on the API interface, dynamically allocates computing resources for each AI model, and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision recommendations in a structured manner in natural language.

[0019] The advantages of this invention are: 1. By collecting multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, and preprocessing each source of industrial data, a multimodal data matrix is ​​constructed, including a structured data matrix, a knowledge graph data matrix, and an image data matrix. Next, a model matrix is ​​built based on the model layer and the agent layer, and the API interface for calling the model layer is set. Then, the input business requirements are obtained and fed into the model matrix. Based on the type of business requirement, the model matrix matches the corresponding agent from the agent layer. The agent matches the corresponding industrial data from the multimodal data matrix based on the business requirements and schedules the model based on the API interface. The AI ​​models adapted within the layer perform collaborative reasoning on industrial data to obtain decision suggestions corresponding to business needs and output these suggestions. In other words, by constructing a unified multimodal data matrix, deep integration and semantic association of multi-source industrial data are achieved, breaking down information silos. Furthermore, through a model matrix driven by business needs, intelligent agents dynamically schedule and coordinate multiple AI models to perform cross-modal joint reasoning, enabling the decision-making process to comprehensively utilize multi-dimensional information such as text, data, and images. This achieves a qualitative leap at the system level from single, isolated analysis to comprehensive, collaborative intelligent decision-making, ultimately greatly improving the accuracy of industrial data decision-making and the level of system intelligence.

[0020] 2. By systematically collecting and preprocessing industrial structured data, document unstructured data, and image unstructured data, a multimodal data matrix (such as a structured data matrix, a knowledge graph data matrix, and an image data matrix) is constructed. This integration capability solves the problem of diverse data sources and heterogeneous formats in industrial environments, avoids the data silo problem in traditional methods, and thus significantly improves the comprehensive utilization rate and consistency of data. For example, by constructing knowledge graphs and enhancing semantic understanding through the BERT model, scattered document data can be associated with structured parameters, providing a more comprehensive data foundation for subsequent decision-making and enhancing the reliability and coverage of the system.

[0021] 3. Through the collaborative design of the model layer (containing various AI models) and the intelligent agent layer (such as intelligent customer service, assisted design, etc.), combined with API interfaces, dynamic matching of business needs and AI models is achieved. The intelligent agent layer acts as a "bridge" and can automatically select the appropriate intelligent agent according to the type of need, and schedule the AI ​​models in the model layer to perform collaborative reasoning. This mechanism not only improves the system's response speed and flexibility, but also avoids resource waste. For example, by dynamically allocating computing resources, it ensures that the system can quickly adapt to changes in complex industrial scenarios (such as fault diagnosis or process optimization) and improve decision-making efficiency.

[0022] 4. The intelligent agent schedules multiple AI models (such as OCR models, language models, CNN models, etc.) based on API interfaces to perform collaborative reasoning on industrial data, and achieves complementarity between models through data transmission. This multi-model collaborative approach can overcome the limitations of a single model. For example, by combining semantic analysis of knowledge graphs and image feature extraction, it can provide more comprehensive and accurate decision-making suggestions in business needs (such as equipment maintenance or product design). At the same time, preprocessing steps (such as outlier removal and feature extraction) further reduce data noise, improve the reliability of reasoning, and make the decision results more practical.

[0023] 5. The construction of the model matrix and the setting of the API interface adopt a modular architecture, separating the model layer from the agent layer and standardizing the interface calls. This design makes it easy for the system to extend to new models or agent types without reconstructing the overall framework. For example, in the context of Industry 4.0, emerging AI technologies can be quickly integrated. In addition, the resource management function ensures the effective use of computing resources, reduces operating costs, and improves the long-term applicability and maintainability of the system, meeting the requirements of industrial systems for stability and evolution.

[0024] 6. Outputting decision recommendations in natural language makes the results easy for industrial personnel to understand and apply. This design reduces the technical threshold and promotes human-machine collaboration. For example, in intelligent customer service or policy matching scenarios, users can directly obtain structured suggestions without the need for professional data analysis knowledge. This not only improves the efficiency of the decision-making process but also enhances the practicality and promotional value of the solution, meeting the usability requirements of the Industrial Internet.

[0025] 7. By constructing a multimodal data matrix, it efficiently integrates industrial structured and unstructured data, and innovatively adopts a collaborative mechanism of model matrix and intelligent agent layer. It can intelligently match AI models and perform collaborative reasoning according to dynamic business needs, which significantly improves the accuracy and adaptability of decision-making. Its modular design and standardized API interface enhance the system's scalability and maintenance efficiency, while natural language output lowers the usage threshold. Ultimately, it realizes comprehensive, intelligent and user-friendly decision support for multi-source heterogeneous industrial data.

[0026] 8. By constructing and mapping multimodal data matrices in a hierarchical manner and adopting layered processing schemes such as structured data time-series storage, the organic integration of multi-source industrial data is achieved, breaking down data silos and significantly improving data integration efficiency and utilization. With the synergy of model layer scheduling capabilities and intelligent agent business docking functions, the model matrix optimizes the adaptability and processing accuracy of AI models through multi-AI model integration and scenario matching mechanisms, breaking through the limitations of single models. Relying on the full-link collaborative architecture of industrial data-AI model-business requirements, value is realized in scenarios such as equipment fault diagnosis and product design, providing core support for the intelligent upgrading of industry. Attached Figure Description

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] Figure 1 This is a flowchart of a multi-source heterogeneous industrial data decision-making method based on a model matrix according to the present invention.

[0029] Figure 2 This is a schematic diagram of the structure of a multi-source heterogeneous industrial data decision system based on a model matrix according to the present invention. Detailed Implementation

[0030] The overall idea of ​​the technical solution in this application is as follows: By constructing a unified multimodal data matrix, deep fusion and semantic association of multi-source industrial data are achieved, breaking down information silos; then, through a model matrix driven by business needs, intelligent agents dynamically schedule and coordinate multiple AI models to perform cross-modal joint reasoning, enabling the decision-making process to comprehensively utilize multi-dimensional information such as text, data, and images, thereby achieving a qualitative change from single, isolated analysis to comprehensive, collaborative intelligent decision-making at the system level, and thus improving the accuracy of industrial data decision-making and the level of system intelligence.

[0031] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the multi-source heterogeneous industrial data decision-making method based on a model matrix according to the present invention includes the following steps: Step S1: Collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data; Step S2: Preprocess the multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix; Step S3: Construct a model matrix based on the model layer and the agent layer, and set the API interface for calling the model layer; Step S4: Obtain the input business requirements (such as equipment fault diagnosis, design optimization suggestions, intelligent customer service), input the business requirements into the model matrix, and the model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirements; Step S5: The intelligent agent matches corresponding industrial data from the multimodal data matrix based on business requirements, performs collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtains decision suggestions corresponding to the business requirements, and outputs the decision suggestions.

[0032] Step S1 specifically involves: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

[0033] Step S2 specifically involves: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation (identifying and removing unreasonable data points through outlier detection and filling missing data using a missing value imputation algorithm). The preprocessed industrial structured data is then stored in a time-series database (such as InfluxDB) with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition (extracting entities such as equipment names and process steps). (In specific implementation, natural language processing toolkits such as NLTK and spaCy can be used.) From the preprocessed unstructured data of each document, knowledge elements including entities, attributes, and relationships are identified. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, a knowledge graph is constructed (stored in a structured form as triples (such as "CNC machine tool, machining, metal parts"). The unstructured data of each document is associated and mapped with the knowledge graph to construct a knowledge graph data matrix (using documents as information source nodes of the knowledge graph, accurately associating document chapters with entity relationships in the graph, and making implicit knowledge of the document explicit). The server performs at least the following steps on the unstructured image data: local feature extraction (convolutional layer), downsampling (pooling layer), and feature integration (fully connected layer) to obtain a fixed-length image feature vector (in practice, a CNN model (such as ResNet-50) can be built based on deep learning frameworks such as TensorFlow and PyTorch). The server also labels the unstructured image data with at least the following tags: category (product type, design style), key design elements (patterns, special structures), and product information to complete the preprocessing of the unstructured image data. Based on the unstructured image data, image feature vectors, and labeled tags, an image data matrix is ​​constructed, forming a complete data structure of "feature vector + text annotation". A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

[0034] Step S3 specifically involves: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

[0035] Step S5 specifically involves: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models (such as fault diagnosis requests: language model + CNN model + multimodal model) in the API interface scheduling model layer, dynamically allocates computing resources for each AI model (adapting to task complexity as needed to avoid idle or overloaded computing resources), and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision suggestions in a structured manner in natural language (e.g., cause of failure + maintenance suggestions / design optimization scheme).

[0036] A preferred embodiment of the multi-source heterogeneous industrial data decision-making system based on a model matrix according to the present invention includes the following modules: The multi-source industrial data acquisition module is used to collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data. A multimodal data matrix construction module is used to preprocess the various multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix. The model matrix construction module is used to construct a model matrix based on the model layer and the agent layer, and to set the API interface for calling the model layer. The business requirement input module is used to obtain input business requirements (such as equipment fault diagnosis, design optimization suggestions, and intelligent customer service), and input the business requirements into the model matrix. The model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirement. The decision suggestion output module is used by the intelligent agent to match corresponding industrial data from the multimodal data matrix based on business needs, perform collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtain decision suggestions corresponding to the business needs, and output the decision suggestions.

[0037] The multi-source industrial data acquisition module is specifically used for: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

[0038] The multimodal data matrix construction module is specifically used for: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation (identifying and removing unreasonable data points through outlier detection and filling missing data using a missing value imputation algorithm). The preprocessed industrial structured data is then stored in a time-series database (such as InfluxDB) with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition (extracting entities such as equipment names and process steps). (In specific implementation, natural language processing toolkits such as NLTK and spaCy can be used.) From the preprocessed unstructured data of each document, knowledge elements including entities, attributes, and relationships are identified. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, a knowledge graph is constructed (stored in a structured form as triples (such as "CNC machine tool, machining, metal parts"). The unstructured data of each document is associated and mapped with the knowledge graph to construct a knowledge graph data matrix (using documents as information source nodes of the knowledge graph, accurately associating document chapters with entity relationships in the graph, and making implicit knowledge of the document explicit). The server performs at least the following steps on the unstructured image data: local feature extraction (convolutional layer), downsampling (pooling layer), and feature integration (fully connected layer) to obtain a fixed-length image feature vector (in practice, a CNN model (such as ResNet-50) can be built based on deep learning frameworks such as TensorFlow and PyTorch). The server also labels the unstructured image data with at least the following tags: category (product type, design style), key design elements (patterns, special structures), and product information to complete the preprocessing of the unstructured image data. Based on the unstructured image data, image feature vectors, and labeled tags, an image data matrix is ​​constructed, forming a complete data structure of "feature vector + text annotation". A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

[0039] The model matrix construction module is specifically used for: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

[0040] The decision suggestion output module is specifically used for: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models (such as fault diagnosis requests: language model + CNN model + multimodal model) in the API interface scheduling model layer, dynamically allocates computing resources for each AI model (adapting to task complexity as needed to avoid idle or overloaded computing resources), and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision suggestions in a structured manner in natural language (e.g., cause of failure + maintenance suggestions / design optimization scheme). In summary, the advantages of this invention are as follows: 1. By collecting multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, and preprocessing each source of industrial data, a multimodal data matrix is ​​constructed, including a structured data matrix, a knowledge graph data matrix, and an image data matrix. Next, a model matrix is ​​built based on the model layer and the agent layer, and the API interface for calling the model layer is set. Then, the input business requirements are obtained and fed into the model matrix. Based on the type of business requirement, the model matrix matches the corresponding agent from the agent layer. The agent matches the corresponding industrial data from the multimodal data matrix based on the business requirements and schedules the model based on the API interface. The AI ​​models adapted within the layer perform collaborative reasoning on industrial data to obtain decision suggestions corresponding to business needs and output these suggestions. In other words, by constructing a unified multimodal data matrix, deep integration and semantic association of multi-source industrial data are achieved, breaking down information silos. Furthermore, through a model matrix driven by business needs, intelligent agents dynamically schedule and coordinate multiple AI models to perform cross-modal joint reasoning, enabling the decision-making process to comprehensively utilize multi-dimensional information such as text, data, and images. This achieves a qualitative leap at the system level from single, isolated analysis to comprehensive, collaborative intelligent decision-making, ultimately greatly improving the accuracy of industrial data decision-making and the level of system intelligence.

[0041] 2. By systematically collecting and preprocessing industrial structured data, document unstructured data, and image unstructured data, a multimodal data matrix (such as a structured data matrix, a knowledge graph data matrix, and an image data matrix) is constructed. This integration capability solves the problem of diverse data sources and heterogeneous formats in industrial environments, avoids the data silo problem in traditional methods, and thus significantly improves the comprehensive utilization rate and consistency of data. For example, by constructing knowledge graphs and enhancing semantic understanding through the BERT model, scattered document data can be associated with structured parameters, providing a more comprehensive data foundation for subsequent decision-making and enhancing the reliability and coverage of the system.

[0042] 3. Through the collaborative design of the model layer (containing various AI models) and the intelligent agent layer (such as intelligent customer service, assisted design, etc.), combined with API interfaces, dynamic matching of business needs and AI models is achieved. The intelligent agent layer acts as a "bridge" and can automatically select the appropriate intelligent agent according to the type of need, and schedule the AI ​​models in the model layer to perform collaborative reasoning. This mechanism not only improves the system's response speed and flexibility, but also avoids resource waste. For example, by dynamically allocating computing resources, it ensures that the system can quickly adapt to changes in complex industrial scenarios (such as fault diagnosis or process optimization) and improve decision-making efficiency.

[0043] 4. The intelligent agent schedules multiple AI models (such as OCR models, language models, CNN models, etc.) based on API interfaces to perform collaborative reasoning on industrial data, and achieves complementarity between models through data transmission. This multi-model collaborative approach can overcome the limitations of a single model. For example, by combining semantic analysis of knowledge graphs and image feature extraction, it can provide more comprehensive and accurate decision-making suggestions in business needs (such as equipment maintenance or product design). At the same time, preprocessing steps (such as outlier removal and feature extraction) further reduce data noise, improve the reliability of reasoning, and make the decision results more practical.

[0044] 5. The construction of the model matrix and the setting of the API interface adopt a modular architecture, separating the model layer from the agent layer and standardizing the interface calls. This design makes it easy for the system to extend to new models or agent types without reconstructing the overall framework. For example, in the context of Industry 4.0, emerging AI technologies can be quickly integrated. In addition, the resource management function ensures the effective use of computing resources, reduces operating costs, and improves the long-term applicability and maintainability of the system, meeting the requirements of industrial systems for stability and evolution.

[0045] 6. Outputting decision recommendations in natural language makes the results easy for industrial personnel to understand and apply. This design reduces the technical threshold and promotes human-machine collaboration. For example, in intelligent customer service or policy matching scenarios, users can directly obtain structured suggestions without the need for professional data analysis knowledge. This not only improves the efficiency of the decision-making process but also enhances the practicality and promotional value of the solution, meeting the usability requirements of the Industrial Internet.

[0046] 7. By constructing a multimodal data matrix, it efficiently integrates industrial structured and unstructured data, and innovatively adopts a collaborative mechanism of model matrix and intelligent agent layer. It can intelligently match AI models and perform collaborative reasoning according to dynamic business needs, which significantly improves the accuracy and adaptability of decision-making. Its modular design and standardized API interface enhance the system's scalability and maintenance efficiency, while natural language output lowers the usage threshold. Ultimately, it realizes comprehensive, intelligent and user-friendly decision support for multi-source heterogeneous industrial data.

[0047] 8. By constructing and mapping multimodal data matrices in a hierarchical manner and adopting layered processing schemes such as structured data time-series storage, the organic integration of multi-source industrial data is achieved, breaking down data silos and significantly improving data integration efficiency and utilization. With the synergy of model layer scheduling capabilities and intelligent agent business docking functions, the model matrix optimizes the adaptability and processing accuracy of AI models through multi-AI model integration and scenario matching mechanisms, breaking through the limitations of single models. Relying on the full-link collaborative architecture of industrial data-AI model-business requirements, value is realized in scenarios such as equipment fault diagnosis and product design, providing core support for the intelligent upgrading of industry.

[0048] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A decision-making method based on multi-source heterogeneous industrial data using a model matrix, characterized in that: Includes the following steps: Step S1: Collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data; Step S2: Preprocess the multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix; Step S3: Construct a model matrix based on the model layer and the agent layer, and set the API interface for calling the model layer; Step S4: Obtain the input business requirements and input the business requirements into the model matrix. The model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirements. Step S5: The intelligent agent matches corresponding industrial data from the multimodal data matrix based on business requirements, performs collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtains decision suggestions corresponding to the business requirements, and outputs the decision suggestions.

2. The multi-source heterogeneous industrial data decision-making method based on a model matrix as described in claim 1, characterized in that: Step S1 specifically involves: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

3. The multi-source heterogeneous industrial data decision-making method based on a model matrix as described in claim 1, characterized in that: Step S2 specifically involves: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation. The preprocessed industrial structured data is then stored in a time-series database with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition. It identifies knowledge elements, including entities, attributes, and relationships, from the preprocessed unstructured data of each document. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, it constructs a knowledge graph and maps the unstructured data of each document to the knowledge graph to construct a knowledge graph data matrix. The server performs at least local feature extraction, downsampling, and feature integration on the unstructured data of each image to obtain a fixed-length image feature vector. It also labels the unstructured data of each image with at least category, key design elements, and product information to complete the preprocessing of the unstructured data of each image. Based on the unstructured data of each image, the image feature vector, and the labeled tags, an image data matrix is ​​constructed. A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

4. The multi-source heterogeneous industrial data decision-making method based on a model matrix as described in claim 1, characterized in that: Step S3 specifically involves: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

5. The multi-source heterogeneous industrial data decision-making method based on a model matrix as described in claim 1, characterized in that: Step S5 specifically involves: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models in the model layer based on the API interface, dynamically allocates computing resources for each AI model, and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision recommendations in a structured manner in natural language.

6. A multi-source heterogeneous industrial data decision-making system based on a model matrix, characterized in that: Includes the following modules: The multi-source industrial data acquisition module is used to collect multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data. A multimodal data matrix construction module is used to preprocess the various multi-source industrial data to construct a multimodal data matrix including a structured data matrix, a knowledge graph data matrix, and an image data matrix. The model matrix construction module is used to construct a model matrix based on the model layer and the agent layer, and to set the API interface for calling the model layer. The business requirement input module is used to obtain input business requirements and input the business requirements into the model matrix. The model matrix matches the corresponding intelligent agent from the intelligent agent layer based on the requirement type of the business requirements. The decision suggestion output module is used by the intelligent agent to match corresponding industrial data from the multimodal data matrix based on business needs, perform collaborative reasoning on the industrial data based on the AI ​​model adapted in the API interface scheduling model layer, obtain decision suggestions corresponding to the business needs, and output the decision suggestions.

7. The multi-source heterogeneous industrial data decision-making system based on a model matrix as described in claim 6, characterized in that: The multi-source industrial data acquisition module is specifically used for: The server collects multi-source industrial data, including structured industrial data, unstructured document data, and unstructured image data, through sensor arrays and databases. The industrial structured data includes at least industrial equipment operating parameters, equipment operation logs, and production orders; the industrial equipment operating parameters include at least temperature, pressure, and rotational speed. The unstructured data in the documents includes at least process manufacturing manuals, technical documents, and product design drawings; The unstructured image data includes at least design drawings, product images, and fault scene images.

8. The multi-source heterogeneous industrial data decision-making system based on a model matrix as described in claim 6, characterized in that: The multimodal data matrix construction module is specifically used for: The server performs preprocessing on each of the industrial structured data, including at least outlier removal and missing value imputation. The preprocessed industrial structured data is then stored in a time-series database with timestamps as indexes and equipment number, data type, and specific numerical value as fields to construct a structured data matrix. The server performs preprocessing on the unstructured data of each document, including at least word segmentation, part-of-speech tagging, and named entity recognition. It identifies knowledge elements, including entities, attributes, and relationships, from the preprocessed unstructured data of each document. After enhancing the semantic understanding of each knowledge element through a pre-trained BERT model, it constructs a knowledge graph and maps the unstructured data of each document to the knowledge graph to construct a knowledge graph data matrix. The server performs at least local feature extraction, downsampling, and feature integration on the unstructured data of each image to obtain a fixed-length image feature vector. It also labels the unstructured data of each image with at least category, key design elements, and product information to complete the preprocessing of the unstructured data of each image. Based on the unstructured data of each image, the image feature vector, and the labeled tags, an image data matrix is ​​constructed. A multimodal data matrix is ​​obtained based on the constructed structured data matrix, knowledge graph data matrix, and image data matrix.

9. A multi-source heterogeneous industrial data decision-making system based on a model matrix as described in claim 6, characterized in that: The model matrix construction module is specifically used for: The server constructs a model matrix based on the model layer and the agent layer, and sets up the API interface for calling the model layer. The model layer includes at least AI models of the types of OCR models, language models, embedding models, CNN models, and multimodal models, and has at least the functions of model scheduling, resource management, and inference; the intelligent agent layer includes at least intelligent agents of the types of intelligent customer service, assisted design, policy matching, and image annotation, serving as a bridge between industrial data, business needs, and AI models.

10. A multi-source heterogeneous industrial data decision-making system based on a model matrix as described in claim 6, characterized in that: The decision suggestion output module is specifically used for: The intelligent agent matches corresponding industrial data from a multimodal data matrix based on business requirements. The industrial data can be structured industrial data, unstructured document data, or unstructured image data. The intelligent agent schedules adapted AI models in the model layer based on the API interface, dynamically allocates computing resources for each AI model, and performs collaborative reasoning on industrial data through each AI model. During the collaborative reasoning process, data is transmitted through the API interface, and finally, decision suggestions corresponding to the business requirements are obtained through reasoning. The intelligent agent outputs decision recommendations in a structured manner in natural language.

Images