Production equipment AI analysis method and equipment based on multi-modal large model, and medium

Abstract

The embodiment of the invention discloses a production equipment AI analysis method and equipment based on a multi-modal large model and a medium, relates to the technical field of artificial intelligence, and is used for solving the problems of low troubleshooting efficiency and low accuracy of production equipment in an existing mode. Identifying physical rule data corresponding to the production equipment; performing enhancement processing on the normal signal of the production equipment based on a preset generative adversarial network to generate joint fault feature data; embedding a preset causal knowledge graph into a hidden layer of the multi-modal large model to train the multi-modal large model according to the joint fault feature data to obtain a trained recognition model; dynamically compressing the identification model according to the resource information of the upper computer to be deployed so as to deploy the processed identification model to the upper computer; and inputting a current signal of the to-be-detected production equipment into the processed identification model so as to output a fault analysis result of the to-be-detected production equipment.

Description

Technical Field

[0001] This specification relates to the technical field of artificial intelligence technology, and particularly to an AI analysis method, device, and medium for production equipment based on a multimodal large model. Background Art

[0002] In the context of the rapid development of today's manufacturing industry, in the fields of modern industrial production and equipment operation and maintenance, the stable and efficient operation of various production equipment plays a decisive role in ensuring the production process, reducing costs, and improving product quality. However, existing production equipment and its automation systems, such as the loading and unloading automation related to production and manufacturing lines including laser cutting machines, manipulators, and material warehouses, have many problems to be solved urgently during operation and after-sales stages, which seriously restricts the performance and wide application of these systems. Therefore, real-time fault detection and handling of production equipment are necessary links to ensure the normal operation of production equipment.

[0003] During the process of fault analysis and handling of production equipment, from the perspective of fault troubleshooting, the structure of production equipment is complex, with multiple systems such as mechanical, electrical, and optical systems working together, which makes fault problems highly complex. Traditional manual troubleshooting methods are inefficient and inaccurate. With the development of artificial intelligence technology, existing traditional machine learning algorithms such as support vector machines, decision trees, and naive Bayes can save the cost of manual on-site monitoring, but they rely on the experience of domain experts and have limited processing capabilities for complex data. Moreover, the structure of existing artificial intelligence models is usually fixed, making it difficult to flexibly adapt to the needs of different equipment and different working conditions. At the same time, limited by the resources of the deployment platform, problems such as the inability to deploy the model or low operating efficiency may occur in actual applications. In addition, although current deep learning models can automatically learn features, their learning capabilities for some complex fault features, especially early weak fault features and multi-factor coupled fault features, are limited, resulting in limited recognition capabilities for complex faults. Summary of the Invention

[0004] To solve the above technical problems, one or more embodiments of this specification provide an AI analysis method, device, and medium for production equipment based on a multimodal large model.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of this specification provide an AI analysis method for production equipment based on a multimodal large model, the method comprising:

[0007] Collecting multi-type data of production equipment to perform physical simulation on the multi-type data to identify the physical law data corresponding to the production equipment;

[0008] Enhance the normal signals of the production equipment based on a pre-set adversarial generative network to generate joint fault feature data; wherein, the adversarial generative network is constrained based on the physical equations corresponding to the physical law data;

[0009] Embed a pre-set causal knowledge graph into the hidden layer of a multi-modal large model to train the multi-modal large model according to the joint fault feature data and obtain a trained recognition model; wherein, the multi-modal large model is a hierarchical hybrid model structure;

[0010] Dynamically compress the recognition model according to the resource information of the to-be-deployed host computer to deploy the processed recognition model to the host computer;

[0011] Input the current signals of the to-be-detected production equipment into the processed recognition model to output the fault analysis result of the to-be-detected production equipment.

[0012] Optionally, in one or more embodiments of this specification, the collecting multi-type data of the production equipment to perform physical simulation on the multi-type data to identify the physical simulation data corresponding to the production equipment specifically includes:

[0013] Collect the initial multi-type data of the production equipment collected by sensors based on a multi-type data interface and preprocess the initial multi-type data to obtain unified multi-type data; wherein, the multi-type data includes: sensor signal data, control instruction data, text log data;

[0014] Perform physical simulation on each fault scenario based on the multi-type data and pre-set fault scenario data to obtain the fault simulation data corresponding to each fault scenario;

[0015] Determine the physical law data corresponding to the production equipment in each fault scenario based on the fault simulation data and the pre-set normal working condition data corresponding to each fault scenario.

[0016] Optionally, in one or more embodiments of this specification, the enhancing the normal signals of the production equipment based on a pre-set adversarial generative network to generate joint fault feature data specifically includes:

[0017] Determine the physical equation corresponding to the physical law data to convert the physical equation into a mathematical constraint corresponding to the pre-set adversarial generative network;

[0018] Input the normal signals of the production equipment and the fault type labels into a generator to process the normal signals based on the multi-modal data corresponding to the fault type labels to obtain the fault feature data corresponding to each fault type label;

[0019] Input the fault feature data into a discriminator with the data constraint as the constraint condition to determine whether the fault feature data conforms to the physical laws of the production equipment;

[0020] If so, obtain the combined fault feature data output by the discriminator.

[0021] Optionally, in one or more embodiments of this specification, before embedding the preset causal knowledge graph into the hidden layer of the multimodal large model to train the multimodal large model according to the combined fault feature data to obtain a trained recognition model, the method further includes:

[0022] Obtain data for each fault description item of the production equipment based on multiple data sources, where the fault description items include: physical entity, fault type, symptom, cause of generation, solution method;

[0023] Based on the fault description items corresponding to the production equipment to be detected, determine the nodes to be constructed corresponding to the preset causal knowledge graph;

[0024] According to the relationship connecting words corresponding to the data of each fault description item, determine the edge relationships between the nodes to be constructed, so as to determine the initial causal knowledge graph according to the nodes to be constructed and the edge relationships; where the edge relationships include: causal relationship, logical relationship, composition relationship, solution relationship;

[0025] Generate a dynamic causal chain of the initial causal knowledge graph based on Granger causality analysis, and expand the initial causal knowledge graph according to the dynamic causal chain to obtain a preset causal knowledge graph.

[0026] Optionally, in one or more embodiments of this specification, the embedding of the preset causal knowledge graph into the hidden layer of the multimodal large model to train the multimodal large model according to the combined fault feature data to obtain a trained recognition model specifically includes:

[0027] Build the architecture of the multimodal large model based on the hierarchical hybrid model structure, and use a graph neural network to encode the preset causal knowledge graph to embed the preset causal knowledge graph into the hidden layer in the architecture to obtain an initial recognition model;

[0028] Train the basic feature extractor of the initial recognition model based on the industrial dataset corresponding to the production equipment, and adjust the initial recognition model according to the combined fault feature data, and update the fusion layer and output layer of the initial recognition model to obtain an updated initial recognition model;

[0029] Optimize the preset classification loss function, generation loss function, and Shapley value regularization corresponding to the initial recognition model based on stochastic gradient descent, and iteratively train and adjust the initial recognition model to obtain the trained recognition model.

[0030] Optionally, in one or more embodiments of this specification, the hierarchical hybrid model structure is trained based on Shapley regularization constraints. The hierarchical hybrid model structure includes: a bottom encoder, a multimodal fusion layer, and an output layer; wherein, the bottom encoder is a CNN or an LSTM, the multimodal fusion layer is composed of a BERT text encoder and a cross-attention mechanism, and the output layer is composed of a fault classification head and a solution generation head.

[0031] Optionally, in one or more embodiments of this specification, dynamically compress the recognition model according to the resource information of the upper computer to be deployed, so as to deploy the processed recognition model to the upper computer, which specifically includes:

[0032] Obtain the resource information of the upper computer to be deployed, and determine the pruning degree of the recognition model based on the resource information and the detection stage requirement data of the production equipment to be detected; wherein, the resource information includes: computing resource information, storage resource information;

[0033] Match the model subnet corresponding to the recognition model based on the pruning degree, and dynamically compress the recognition model based on the model subnet;

[0034] Obtain the model parameter type corresponding to the compressed recognition model, and convert the model parameter type into a low-precision data type to obtain the processed recognition model;

[0035] Upload the processed recognition model to the specified directory of the upper computer, and establish a connection between the processed recognition model and the production equipment to be detected to realize the deployment of the processed recognition model.

[0036] Optionally, in one or more embodiments of this specification, input the current signal of the production equipment to be detected into the processed recognition model to output the fault analysis result of the production equipment to be detected, which specifically includes:

[0037] Collect the current signal of the production equipment to be detected, and align the current signal based on the time stamp to obtain the input signal;

[0038] Input the input signal into the processed recognition model, and based on the inference of the processed recognition model, output the fault analysis result corresponding to the fault description item of the production equipment to be detected.

[0039] One or more embodiments of this specification provide a production equipment AI analysis device based on a multi-modal large model. The device includes:

[0040] At least one processor; and,

[0041] A memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores instructions executable by the at least one processor. When the instructions are executed by the at least one processor, the at least one processor is enabled to: execute any of the above-mentioned methods.

[0043] A non-volatile computer storage medium provided by one or more embodiments of this specification stores computer-executable instructions, and the computer-executable instructions are configured to: be capable of executing any of the above-mentioned methods.

[0044] The above-mentioned at least one technical solution adopted by the embodiments of this specification can achieve the following beneficial effects:

[0045] Collecting various types of data of production equipment and performing physical simulation can deeply explore the physical law data followed by the equipment operation, which helps to make the subsequent judgment of equipment faults more scientific and accurate. Using a pre-set adversarial generation network to enhance the normal signal to generate joint fault feature data, and the adversarial generation network is constrained based on the physical equation corresponding to the physical law data. The generated joint fault feature data not only contains the information of the normal operation of the equipment, but also highlights the possible fault features, while ensuring the consistency between the data and the actual physical laws of the equipment, improving the data's ability to represent faults, and avoiding the problems of traditional machine learning relying on manual feature extraction, being limited by expert experience and having low efficiency, as well as the problem that deep learning's automatic feature extraction has insufficient learning ability for complex fault features. Embedding a pre-set causal knowledge graph into the hidden layer of the multi-modal large model enables the model to fuse domain knowledge and multi-modal data information, solving the problem that existing models cannot locate the root cause. The hierarchical hybrid model structure of the multi-modal large model can make full use of feature representations at different levels to better learn complex patterns in the data. Dynamically compressing the recognition model according to the resource information of the upper computer to be deployed can not only avoid resource waste but also ensure that the model can operate efficiently on devices with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0047] Figure 1 It is a schematic flowchart of a method for AI analysis of production equipment based on a multimodal large model provided by an embodiment of this specification;

[0048] Figure 2 It is a simulation time series diagram of the over-temperature of the focusing lens of a laser cutting machine provided by an embodiment of this specification;

[0049] Figure 3 It is an example diagram of data generation with a pre-set adversarial generation network provided by an embodiment of this specification;

[0050] Figure 4 It is an example diagram of the result of fault analysis provided by an embodiment of this specification;

[0051] Figure 5 It is a reasoning flowchart of a pre-set causal knowledge graph provided by an embodiment of this specification;

[0052] Figure 6 It is a bar chart of feature contribution degree provided by an embodiment of this specification;

[0053] Figure 7 It is a comparison chart of the diagnostic accuracy rate of an identification model provided by an embodiment of this specification;

[0054] Figure 8 It is a distribution chart of response time provided by an embodiment of this specification;

[0055] Figure 9 It is a schematic structural diagram of an AI analysis device for production equipment based on a multimodal large model provided by an embodiment of this specification;

[0056] Figure 10 It is a schematic structural diagram of a non-volatile storage medium provided by an embodiment of this specification. Detailed implementation manners

[0057] The embodiments of this specification provide a method, device, and medium for AI analysis of production equipment based on a multimodal large model.

[0058] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this specification.

[0059] As Figure 1 shown, the embodiment of this specification provides a schematic flowchart of a method for AI analysis of production equipment based on a multi-modal large model. It can be Figure 1 seen that in one or more embodiments of this specification, a method for AI analysis of production equipment based on a multi-modal large model specifically includes the following steps:

[0060] S101: Collect multi-type data of the production equipment to perform physical simulation on the multi-type data to identify the physical law data corresponding to the production equipment.

[0061] In order to provide a solid theoretical basis for fault diagnosis and treatment, and at the same time avoid the problem that the existing artificial intelligence methods usually only process single-type data, such as only using the vibration data or temperature data of the equipment for analysis, which cannot fully utilize the multi-dimensional information generated during the operation of the equipment, resulting in an incomplete understanding of the equipment status, that is, the limitation problem of single data type analysis, and improve the comprehensiveness of production equipment fault analysis. In the embodiments of this specification, a multi-type data interface is used to collect initial multi-type data such as sensor signal data, control instruction data, and text log data of the production equipment, and then preprocessing is performed to unify the data format for subsequent analysis. For example, convert data with different units and frequencies collected by different sensors into data with a unified standard. Then, combine the collected multi-type data with the preset fault scenario data to perform physical simulation on various fault scenarios, so as to find out the physical law data corresponding to the production equipment in each fault scenario. In this process, by uniformly processing multi-type data and combining physical simulation analysis, the limitation of single data type analysis is avoided, and the equipment status is comprehensively and comprehensively reflected. By determining the physical law data through physical simulation, the internal operation law of the equipment in different fault scenarios can be deeply explored, and the essential reasons for the occurrence and development of faults can be understood.

[0062] Specifically, in one or more embodiments of this specification, collecting multi-type data of the production equipment to perform physical simulation on the multi-type data to identify the physical simulation data corresponding to the production equipment specifically includes:

[0063] Collect the initial multi-type data of the production equipment collected by the sensors through multi-type data interfaces. Since there are differences in data formats, units, frequencies, etc., it is necessary to preprocess the initial multi-type data to obtain the unified multi-type data. Among them, it should be noted that the multi-type data includes: sensor signal data, control instruction data, and text log data. Then, through the multi-type data and the preset fault scenario data, physical simulations are carried out for each fault scenario to obtain the fault simulation data corresponding to each fault scenario. For example, faults such as abnormal attenuation of laser power and deviation of cutting head positioning are simulated to generate corresponding fault simulation data, so as to reflect the changes of various physical quantities of the production equipment during faults. Then, based on the fault simulation data and the preset normal working condition data corresponding to each fault scenario, the physical law data of the production equipment corresponding to each fault scenario is determined. That is, by comparing the fault simulation data with the preset normal working condition data corresponding to each fault scenario, analyzing the differences, and extracting the change laws, mutual relationships of physical quantities and the deviation degree from the normal state during the occurrence of faults, so as to determine the physical law data of the production equipment under different fault scenarios. This specification will be further illustrated by taking a laser cutting production equipment as an example, as Figure 2 shown, it is a simulation time sequence diagram of the temperature exceeding the limit of the focusing lens of a laser cutting machine provided by an embodiment of this specification. Based on the simulation results, it can be seen that under normal working conditions, the temperature is stable at 70-75°C, and under fault conditions, the temperature linearly rises from 75°C to 90°C (slope 0.25°C / s). After repair, the temperature drops to 72°C after 300 seconds (restored after cleaning the filter screen). The key parameter is the fault trigger condition: the cooling water flow rate < 5 L / min (normal value 10 L / min), and the model response time is 187 ms to output an alarm after detecting that the temperature > 80°C.

[0064] In this process, by comprehensively analyzing multi-type data, the limitations of single data can be avoided, the equipment status can be comprehensively grasped, and the accuracy of fault diagnosis can be improved. By comparing with the normal working condition data, the physical mechanism and internal law of the fault occurrence can be explored, providing a theoretical basis for fault diagnosis and treatment. The physical law data determined based on physical simulation can be used as a reference standard and characteristic index for fault diagnosis, which helps to identify and analyze the faults of production equipment in the future.

[0065] S102: Based on the preset generative adversarial network, enhance the normal signals of the production equipment to generate joint fault feature data; among them; the generative adversarial network is constrained based on the physical equations corresponding to the physical law data.

[0066] Although current deep learning models can automatically learn features, their learning ability for some complex fault features, especially early weak fault features and multi-factor coupled fault features, is limited. Therefore, in order to better generate representative joint fault features, improve the ability to identify complex faults, and obtain data related to the faults of production equipment while avoiding the problem that the data generated by simply using GAN lacks physical rationality, in the embodiments of this specification, physical equations corresponding to physical law data such as laser cutting thermo-mechanical coupling equations are introduced as constraint conditions in the generation training of the adversarial generation network for limitation. Thus, the normal signals of the production equipment can be enhanced by the preset adversarial generation network to generate joint fault feature data such as Figure 3 as shown. Through the joint fault feature data generated based on physical laws in this process, the recognition model obtained by subsequent training can better handle various complex situations. At the same time, the adversarial generation network with physical equation constraints can generate fault feature data close to the real physical situation through the enhancement processing of normal signals, supplementing the shortage of fault data to a certain extent. This enables the model not to rely on a large amount of real fault data during training, reducing the data collection cost, and also ensuring the performance and reliability of the model.

[0067] Specifically, in one or more embodiments of this specification, enhancing the normal signals of the production equipment by the preset adversarial generation network to generate joint fault feature data specifically includes:

[0068] During the operation of the production equipment, there are various physical laws, such as thermo-mechanical coupling, mechanical motion, etc., which are described by corresponding physical equations. Therefore, in order to enable the network to follow the real physical characteristics of the equipment when generating data, the physical equations corresponding to the physical law data will be determined, and then the physical equations will be converted into mathematical constraints corresponding to the preset adversarial generation network. Then, the normal signals of the production equipment and the fault type labels are input into the generator. Based on the multi-modal data corresponding to the fault type labels, the normal signals are processed to simulate and obtain the fault feature data corresponding to each fault type label. The fault feature data generated by the generator is input into the discriminator, and the discriminator uses the previously converted mathematical constraints as conditions to judge whether these fault feature data conform to the physical laws of the production equipment. When the discriminator determines that the fault feature data conforms to the physical laws, these data are output as joint fault feature data. These data integrate various fault features and can more comprehensively reflect the state of the production equipment under different fault conditions, providing rich and accurate data support for subsequent fault diagnosis using multi-modal large models.

[0069] S103: Embed the pre - set causal knowledge graph into the hidden layer of the multi - modal large model to train the multi - modal large model according to the joint fault feature data, and obtain a trained recognition model; wherein, the multi - modal large model is a hierarchical hybrid model structure.

[0070] Most existing deep learning models are black - box models, making it difficult to explain the basis and process of the model's decision - making. This is a serious problem in the field of equipment fault diagnosis where high requirements are placed on safety and reliability. Therefore, in order to increase the interpretability of the model so that the trained recognition model can infer fault situations based on causal relationships and multi - modal information, and at the same time improve the performance and accuracy of the model, this specification will embed the pre - set causal knowledge graph into the hidden layer of the multi - modal large model to train the multi - modal large model according to the joint fault feature data obtained in the above steps, and obtain a trained recognition model, thus solving the problem that the existing model can only output the fault type and cannot locate the root cause. Among them, it should be noted that the multi - modal large model is a hierarchical hybrid model structure. The hierarchical hybrid model structure includes: a bottom - layer encoder, a multi - modal fusion layer, and an output layer; wherein, the bottom - layer encoder is a CNN or an LSTM, the multi - modal fusion layer is composed of a BERT text encoder and a cross - attention mechanism, and the output layer is composed of a fault classification head and a solution generation head. Specifically, text encoding: The BERT model extracts the semantic vector of the alarm log (e.g., "E006" → embedding dimension 742). Cross - attention mechanism: Calculate the correlation weight between sensor features and text features. Output layer: Fault classification head: Softmax outputs the probability of the fault type. Solution generation head: The Transformer decoder generates natural - language steps (e.g., "Step 1: Check the connection of the wireless handle receiver"). Strategy: Two - stage fine - tuning: Pre - training stage: Train the basic feature extractor on a general industrial dataset (such as NASA bearing data). Domain adaptation stage: Fine - tune with laser - cutting exclusive data, freeze the CNN / LSTM bottom - layer, and only update the fusion layer and the output layer. Loss function: The classification loss is cross - entropy, and the generation loss is the BLEU score for solution text quality. Interpretability constraint: Shapley value regularization to ensure that the weights of key features are traceable.

[0071] Further, in one or more embodiments of this specification, before embedding the pre - set causal knowledge graph into the hidden layer of the multi - modal large model to train the multi - modal large model according to the joint fault feature data and obtain a trained recognition model, the method further includes the following process:

[0072] First, obtain the data of each fault description item of the production equipment based on multiple data sources, where, for example Figure 4The described fault description items may include: physical entities, fault types, symptoms, causes, solutions, etc. Then, according to the fault description items corresponding to the production equipment to be detected, the nodes to be constructed corresponding to the pre-set causal knowledge graph are determined. According to the relationship connecting words corresponding to the data of each fault description item, the edge relationships between the nodes to be constructed are determined, so as to determine the initial causal knowledge graph based on the nodes to be constructed and the edge relationships; among them, it should be noted that the edge relationships include: causal relationship, logical relationship, composition relationship, solution relationship. As Figure 5 As shown in the inference flow chart of a pre-set causal knowledge graph, if "cooling water flow rate decreases" is earlier than "temperature rises", it is determined as a cooling system fault, and then the pump voltage is monitored to finally determine whether the problem is the filter or the pump. If "temperature rises" is earlier than "cooling water flow rate decreases", it is suspected that the laser head lens is contaminated. After obtaining the initial causal knowledge graph, a dynamic causal chain of the initial causal knowledge graph will be generated based on Granger causal analysis, so as to expand the initial causal knowledge graph according to the dynamic causal chain and obtain the pre-set causal knowledge graph. For example, in an application scenario, analyze the Granger causal relationship between variables: if "cooling water flow rate decreases" is earlier than "temperature rises", it is determined as a cooling system fault, and then the pump voltage is monitored to finally determine whether the problem is the filter or the pump. If "temperature rises" is earlier than "cooling water flow rate decreases", it is suspected that the laser head lens is contaminated. Dynamically generate a causal chain (such as "E102 alarm → low cooling water flow rate → filter blockage"), and at this time, the root cause localization accuracy can be improved from 67% to 89%.

[0073] Specifically, in one or more embodiments of this specification, the pre-set causal knowledge graph is embedded in the hidden layer of the multi-modal large model to train the multi-modal large model according to the joint fault feature data, and the trained recognition model is obtained, which specifically includes the following processes:

[0074] The existing artificial intelligence model structure is fixed and difficult to adapt to different equipment and working conditions. Therefore, in the embodiment of this specification, the architecture of the multimodal large model is built based on the hierarchical hybrid model structure, and the preset causal knowledge graph is encoded using the graph neural network, so that the preset causal knowledge graph is embedded in the hidden layer of the architecture to obtain the initial recognition model. Then, the basic feature extractor of the initial recognition model is trained based on the industrial data set corresponding to the production equipment, and the initial recognition model is adjusted according to the joint fault feature data to update the fusion layer and output layer of the initial recognition model to obtain the updated initial recognition model. Then, the preset classification loss function, generation loss function and Shapley value regularization corresponding to the initial recognition model are optimized based on random gradient descent, and the initial recognition model is iteratively trained and adjusted to obtain the trained recognition model. In this process, the preset causal knowledge graph is encoded using the graph neural network and embedded in the hidden layer of the multimodal large model, which can integrate the prior domain knowledge and causal relationship into the model. In this way, the model can better explore the potential connection between data based on this knowledge when processing data. In addition, a multimodal large model is built based on a hierarchical hybrid model structure, so that the model can process information at different levels in a hierarchical manner. Adjusting the fusion layer and output layer of the initial recognition model according to the joint fault feature data can make the model better adapt to the specific fault feature combination, optimize the fusion method and output results of different features in the model, and make the model more accurate in judging various fault conditions.

[0075] It should be noted that the above Shapley value analysis process will calculate Figure 6 The contribution of each input feature to fault classification is shown, for example: "temperature sensor reading" contribution: 42.3%, "water pump voltage monitoring" contribution: 28.7%, "cooling water flow" contribution: 19.5, alarm log "E102" contribution: 9.5%. Rule engine verification: compare the model output results with our rule base, and trigger a review if there is a conflict. For example, the model recommends "replace the water pump" but the water pump voltage monitoring is normal, then prompt manual inspection, based on this process to improve the accuracy of troubleshooting.

[0076] S104: Dynamically compressing the recognition model according to resource information of the host computer to be deployed, so as to deploy the processed recognition model to the host computer.

[0077] The resource limitations of the deployment platform are often ignored when developing existing artificial intelligence models, which may lead to deployment difficulties or low operating efficiency. Therefore, in order to achieve millisecond-level response on a resource-constrained host computer, the embodiment of this specification will dynamically compress the recognition model according to the resource information of the host computer to be deployed, so that the processed recognition model is deployed to the host computer. By lightweight processing of the recognition model, the resource consumption of the host computer is reduced, thereby improving the response speed of the host computer.

[0078] Specifically, in one or more embodiments of this specification, dynamically compressing the recognition model according to the resource information of the upper computer to be deployed, so as to deploy the processed recognition model to the upper computer specifically includes:

[0079] Obtain the resource information of the upper computer to be deployed, and determine the pruning degree of the recognition model based on the resource information and the detection stage requirement data of the production equipment to be detected; among them, it should be noted that the resource information includes: computing resource information and storage resource information. Then, match the model subnet corresponding to the recognition model based on the above pruning degree, and thus dynamically compress the recognition model according to the model subnet. For example: select the model subnet according to the current processing stage, and the high-precision mode can be enabled in the finishing stage, and the energy-saving mode can be switched to in the idle stage.

[0080] Obtain the model parameter type corresponding to the compressed recognition model, and convert the model parameter type into a low-precision data type to obtain the processed recognition model. For example: when converting the FP32 model to INT8, the inference speed can be accelerated by 3 times. Then, upload the processed recognition model to the specified directory of the upper computer, and establish a connection between the processed recognition model and the production equipment to be detected to realize the deployment of the processed recognition model.

[0081] By obtaining the computing resources and storage resource information of the upper computer to be deployed and combining the requirement data of the production equipment detection stage to determine the pruning degree of the recognition model, the model can be optimized according to the actual resource status of the upper computer, avoiding the situation that the model cannot be deployed due to insufficient resources or resource waste, and making the model accurately adapted to the hardware resources. Dynamically compressing based on the pruning degree to match the model subnet corresponding to the recognition model can flexibly adjust the model structure to adapt to different resource environments while ensuring the model performance. For an upper computer with limited resources, a smaller subnet can be selected to reduce computing and storage requirements; for an upper computer with sufficient resources, a larger subnet can be selected to obtain better model performance. Uploading the processed recognition model to the specified directory of the upper computer and establishing a connection with the production equipment to be detected realizes the rapid deployment of the model. This standardized deployment process is simple and easy to implement, reduces the possibility of manual intervention and configuration errors, and improves the efficiency and reliability of model deployment.

[0082] S105: Input the current signal of the production equipment to be detected into the processed recognition model to output the fault analysis result of the production equipment to be detected.

[0083] After obtaining the processed recognition model based on the above step S104, the current signal of the production equipment to be detected will be input into the processed recognition model, so as to output the fault analysis result of the production equipment to be monitored. Specifically, in one or more embodiments of this specification, inputting the current signal of the production equipment to be detected into the processed recognition model to output the fault analysis result of the production equipment to be detected specifically includes: collecting the current signal of the production equipment to be detected, and aligning the current signal based on the timestamp to obtain the input signal. Then, input the input signal into the processed recognition model, and thus, according to the inference of the processed recognition model, output the fault analysis result corresponding to the fault description item of the production equipment to be detected. For example: when the input data is "Vacuum pressure sensor: 20 kPa, Text log: 'Feeding timeout: workpiece not in place'", the fault analysis result output by the model is: "1. Problem cause: Vacuum pipeline leakage or suction cup seal ring aging. 2. Solution: Emergency disposal: Switch to the spare suction cup. Fundamental repair: Use a leak detector to locate the leakage point and replace the seal ring. Step process: 1. Close the gas pipeline valve; 2. Apply soapy water to detect air bubbles in the pipeline; 3. Disassemble the suction cup and check the wear degree of the seal ring."

[0084] As Figure 7 shown is a comparison chart of the diagnostic accuracy rate of an identification model provided by an embodiment of this specification. Based on Figure 7 it can be known that the fault handling method provided in the embodiments of this specification has a more accurate fault detection accuracy rate compared with the traditional model system and the existing LSTM+CNN method, and as Figure 8 shown, based on its response time distribution chart, it can be determined that the average response time at the edge end is 182 ms, meeting the industrial standard of real-time detection. And after converting the FP32 model to the INT8 model, the power consumption and response time are significantly reduced.

[0085] As Figure 9 shown, an embodiment of this specification provides a schematic structural diagram of a production equipment AI analysis device based on a multi-modal large model. From Figure 9 it can be known that in one or more embodiments of this specification, a production equipment AI analysis device based on a multi-modal large model, the device includes:

[0086] At least one processor; and,

[0087] A memory communicatively connected to the at least one processor; wherein,

[0088] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: execute any one of the above methods.

[0089] As Figure 10As shown, the embodiments of this specification provide a structural schematic diagram of a non-volatile storage medium. From Figure 10 it can be seen that in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 1001, and the computer-executable instructions 1001 can execute any of the above-mentioned methods.

[0090] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0091] The above specifically describes certain embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] The above is only one or more embodiments of this specification and is not used to limit this specification. For those skilled in the art, one or more embodiments of this specification can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of this specification.

[0093] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0094] The above specifically describes certain embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0095] The above description is only for one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of this specification.

Claims

1. An AI analysis method for production equipment based on a multimodal large model, characterized in that, The method includes: Collecting multi-type data of each production device in the current scenario to perform physical simulation on the multi-type data to identify the physical law data corresponding to the production device; Based on a pre-set adversarial generation network, enhancing the normal signals of the production device to generate joint fault feature data; wherein, the adversarial generation network is constrained based on the physical equation corresponding to the physical law data; Embedding a pre-set causal knowledge graph into the hidden layer of a multi-modal large model to train the multi-modal large model according to the joint fault feature data to obtain a trained recognition model; wherein, the multi-modal large model is a hierarchical hybrid model structure; Dynamically compressing the recognition model according to the resource information of the to-be-deployed host computer to deploy the processed recognition model to the host computer; Inputting the current signal of the production device to be detected into the processed recognition model to output the fault analysis result of the production device to be detected.

2. The AI analysis method for a production device based on a multi-modal large model according to claim 1, wherein, The collecting multi-type data of the production device to perform physical simulation on the multi-type data to identify the physical simulation data corresponding to the production device specifically includes: Collecting the initial multi-type data of the production device collected by the sensor based on a multi-type data interface and preprocessing the initial multi-type data to obtain unified multi-type data; wherein, the multi-type data includes: sensor signal data, control instruction data, text log data; Performing physical simulation on each fault scenario based on the multi-type data and pre-set fault scenario data to obtain the fault simulation data corresponding to each fault scenario; Based on the fault simulation data and the pre-set normal working condition data corresponding to each fault scenario, determining the physical law data corresponding to the production device in each fault scenario.

3. The AI analysis method for a production device based on a multi-modal large model according to claim 1, wherein, The enhancing the normal signals of the production device based on a pre-set adversarial generation network to generate joint fault feature data specifically includes: Determining the physical equation corresponding to the physical law data to convert the physical equation into a mathematical constraint corresponding to the pre-set adversarial generation network; Inputting the normal signal and the fault type label of the production device into a generator to process the normal signal based on the multi-modal data corresponding to the fault type label to obtain the fault feature data corresponding to each fault type label; Inputting the fault feature data into a discriminator with the data constraint as the constraint condition to determine whether the fault feature data conforms to the physical law of the production device; If so, obtaining the joint fault feature data output by the discriminator.

4. The AI analysis method for a production device based on a multimodal large model according to claim 1, wherein, Before the embedding the pre-set causal knowledge graph into the hidden layer of the multi-modal large model to train the multi-modal large model according to the joint fault feature data to obtain a trained recognition model, the method further includes: Obtaining the data of each fault description item of the production device based on multiple data sources, wherein the fault description items include: physical entity, fault type, symptom, cause of generation, solution method; Based on the fault description items corresponding to the production device to be detected, determining the nodes to be constructed corresponding to the pre-set causal knowledge graph; Determine the edge relationships between the to-be-constructed nodes according to the relational connectives corresponding to the data of each fault description item, so as to determine the initial causal knowledge graph based on the to-be-constructed nodes and the edge relationships; wherein, the edge relationships include: causal relationship, logical relationship, composition relationship, solution relationship; Generate the dynamic causal chain of the initial causal knowledge graph based on Granger causality analysis, so as to expand the initial causal knowledge graph according to the dynamic causal chain and obtain the preset causal knowledge graph.

5. The AI analysis method for a production device based on a multimodal large model according to claim 1, wherein Embedding the preset causal knowledge graph into the hidden layer of the multimodal large model, and training the multimodal large model according to the joint fault feature data to obtain the trained recognition model, specifically including: Build the architecture of the multimodal large model based on the hierarchical hybrid model structure, and use the graph neural network to encode the preset causal knowledge graph, so as to embed the preset causal knowledge graph into the hidden layer in the architecture and obtain the initial recognition model; Train the basic feature extractor of the initial recognition model based on the industrial data set corresponding to the production equipment, and adjust the initial recognition model according to the joint fault feature data, and update the fusion layer and output layer of the initial recognition model to obtain the updated initial recognition model; Optimize the preset classification loss function, generation loss function and Shapley value regularization corresponding to the initial recognition model based on stochastic gradient descent, and iteratively train and adjust the initial recognition model to obtain the trained recognition model.

6. The AI analysis method for a production device based on a multi-modal large model according to claim 1, characterized in that, The hierarchical hybrid model structure is trained based on Shapley regularization constraints. The hierarchical hybrid model structure includes: a bottom encoder, a multimodal fusion layer and an output layer; wherein, the bottom encoder is a CNN or an LSTM, the multimodal fusion layer is composed of a BERT text encoder and a cross-attention mechanism, and the output layer is composed of a fault classification head and a solution generation head.

7. A production equipment AI analysis method based on a multimodal large model according to claim 1, characterized in that, Dynamically compress the recognition model according to the resource information of the to-be-deployed host computer, so as to deploy the processed recognition model to the host computer, specifically including: Obtain the resource information of the to-be-deployed host computer, so as to determine the pruning degree of the recognition model based on the resource information and the detection stage requirement data of the to-be-detected production equipment; wherein, the resource information includes: computing resource information, storage resource information; Match the model subnet corresponding to the recognition model based on the pruning degree, so as to dynamically compress the recognition model based on the model subnet; Obtain the model parameter type corresponding to the compressed recognition model, so as to convert the model parameter type into a low-precision data type to obtain the processed recognition model; Upload the processed recognition model to the specified directory of the host computer, and establish the connection between the processed recognition model and the to-be-detected production equipment to realize the deployment of the processed recognition model.

8. The AI analysis method for a production device based on a multi-modal large model according to claim 1, wherein, Input the current signal of the to-be-detected production equipment into the processed recognition model to output the fault analysis result of the to-be-detected production equipment, specifically including: Collect the current signals of the production equipment to be detected, and align the current signals based on timestamps to obtain input signals; Input the input signals into the processed recognition model, and based on the inference of the processed recognition model, output a fault analysis result corresponding to the fault description items of the production equipment to be detected.

9. A production equipment AI analysis device based on a multi-modal large model, comprising: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: execute the method according to any one of claims 1-8 above.

10. A non-volatile storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions can: execute the method according to any one of claims 1-8 above.

Images