A method and system for industrial field safety management and control combined with multimodal models

Abstract

The present application relates to the field of safety management and control technology, and discloses an industrial safety management and control method and system combined with a multimodal model, including: obtaining multimodal data in the industrial production process; extracting industrial equipment, risks and operating steps as keyword entities from the multimodal data, and constructing an entity map by analyzing the association relationship between the keyword entities; identifying the hazardous factors in the production process based on the multimodal data through a preset hazard identification model; screening out target entity chains matching the hazardous factors in the entity map through a preset reordering model; matching corresponding hazard response plans in a preset safety operation knowledge base based on the target entity chain to perform safety management and control in the industrial field; the present application can improve the accuracy of hazard source identification and the efficiency of control measure generation, thereby providing a more scientific and efficient solution for safety management and control.

Description

Technical Field

[0001] The present invention relates to the field of security management and control technology, and in particular to an industrial security management and control method and system combining a multimodal model. Background Art

[0002] In the industrial field, safety management and control can reduce the accident rate by identifying potential danger information in advance through real-time monitoring and analysis of data such as equipment operating status, personnel operating behavior, and process parameters. However, with the increasing complexity and degree of automation in industrial production, the traditional single safety management and control method based on single-modal data has become difficult to cope with new risk challenges.

[0003] The existing technology has the following problems: data based on a single modality cannot reflect the comprehensive industrial production status, resulting in a high risk omission rate in complex scenarios; direct matching through keywords cannot capture the deep semantic association relationship between entities, resulting in poor correlation between the retrieved relevant response measures; a single response plan formulation method is difficult to cope with the dynamic changes of industrial scenarios, resulting in poor results of the formulated plans; in order to solve at least one of the above problems, the present invention proposes an industrial field safety management and control method and system combined with a multimodal model. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the main purpose of the present invention is to provide an industrial field safety management method and system combined with a multimodal model, which can effectively solve the problems in the background technology. The specific technical solutions of the present invention are as follows:

[0005] A multimodal model-based industrial safety management and control method includes:

[0006] Acquire multimodal data from industrial production processes;

[0007] Extracting industrial equipment, risks, and operating steps from the multimodal data as keyword entities, respectively, and constructing an entity graph by analyzing the association relationships between the keyword entities;

[0008] Identifying hazardous factors in the production process using a preset hazard identification model based on the multimodal data;

[0009] Using a preset re-ranking model, a target entity chain matching the risk factor is screened out from the entity graph;

[0010] According to the target entity chain, the corresponding hazard response plan is matched in the preset safety operation knowledge base to perform safety management and control in the industrial field.

[0011] Specifically, extracting industrial equipment, risks, and operating steps from the multimodal data as keyword entities, and constructing an entity graph by analyzing the association relationship between the keyword entities includes:

[0012] Inputting the multimodal data into a preset keyword extraction model respectively, extracting industrial equipment, risks, and operating steps as keyword entities;

[0013] Divide the keyword entities into different levels according to industrial equipment, risks, and operating steps to obtain multi-layer entities;

[0014] By analyzing the relationship between entities at each layer, an entity graph is constructed.

[0015] Specifically, the entity graph is constructed by analyzing the association relationship between entities in each layer, wherein the multi-layer entities include a physical layer, a logical layer, and an operational layer, including:

[0016] By analyzing the operating status of physical layer entities and the risk triggering probability between logical layer entities, risk association is obtained;

[0017] By analyzing the risk response relationship between the logical layer entities and the operational layer entities, the response association is obtained;

[0018] An association is established between the corresponding physical layer entity and the logical layer entity according to the risk association, and an association is established between the corresponding logical layer entity and the operation layer entity according to the response association, to obtain an entity graph.

[0019] Specifically, identifying hazardous factors in the production process based on the multimodal data using a preset hazard identification model includes:

[0020] From the multimodal data, the features of the corresponding modal data are extracted respectively and the joint feature vector is constructed;

[0021] According to the joint feature vector, the dangerous conditions in the production process are identified through a preset danger identification model to obtain the dangerous factors.

[0022] Specifically, the method of screening out a target entity chain matching the risk factor in the entity graph by using a preset reordering model includes:

[0023] By analyzing the association between the risk factors and each entity in the entity map, a correlation matrix is ​​constructed;

[0024] According to the association matrix, entities are screened in each layer of the entity graph through a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain.

[0025] Specifically, the association matrix is ​​constructed by analyzing the association relationship between the risk factors and each entity in the entity graph, including:

[0026] Extract corresponding features from the dangerous factors and construct the first feature vector;

[0027] According to the state of each entity in the entity graph, each entity is mapped into a second feature vector;

[0028] Obtaining the correlation between the risk factor and each entity by calculating the similarity between the first feature vector and the second feature vector;

[0029] constructing an initial correlation matrix according to the correlation degree;

[0030] Within a preset time period, the hazardous factors and the changes in the industrial production process environment of the entity are analyzed respectively, and the initial correlation matrix is ​​updated to obtain a correlation matrix.

[0031] Specifically, according to the association matrix, entities are screened in each layer of the entity graph by using a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain, including:

[0032] Extract the correlation degree corresponding to each layer of entities in the entity graph from the correlation matrix to obtain the correlation submatrix of each layer of entities;

[0033] According to the correlation submatrix, entities with a correlation degree greater than a preset correlation degree threshold are screened out as a candidate entity set for each layer;

[0034] In combination with the connection relationship between entities in the entity graph, a plurality of connected entity chains are screened out from the candidate entity set to obtain a candidate entity chain set;

[0035] The preset reordering model is used to analyze the association degree of each entity chain in the candidate entity chain set, calculate the reordering score, and select the entity chain with the highest reordering score as the target entity chain.

[0036] Specifically, the target entity chain is matched with corresponding hazard response plans in a preset safety operation knowledge base to perform safety management and control in the industrial field, including:

[0037] According to the target entity chain, the system matches the preset security operation knowledge base in hierarchical order to obtain the corresponding response measures;

[0038] Combined with the correlation between entities in the target entity chain, the response measures are dynamically combined to obtain corresponding hazard response plans to conduct safety management and control in the industrial field.

[0039] Specifically, the response measures are dynamically combined based on the correlation between entities in the target entity chain to obtain corresponding hazard response plans to perform safety management and control in the industrial field, including:

[0040] According to the target entity chain, the total correlation between the corresponding entity and the connected entities is extracted from the correlation matrix;

[0041] According to the total relevance, assign a corresponding priority weight to each entity in the target entity chain;

[0042] According to the priority weights, the corresponding response measures are dynamically combined to obtain corresponding hazard response plans to conduct safety management and control in the industrial field.

[0043] An industrial field safety management and control system combining a multimodal model, used to implement the industrial field safety management and control method combining a multimodal model, comprising:

[0044] Data acquisition module, which acquires multimodal data from industrial production processes;

[0045] An entity graph construction module extracts industrial equipment, risks, and operating steps from the multimodal data as keyword entities, and constructs an entity graph by analyzing the association relationship between the keyword entities;

[0046] A hazardous factor identification module, which identifies hazardous factors in the production process based on the multimodal data and a preset hazardous identification model;

[0047] An entity chain screening module, which screens target entity chains matching the risk factors in the entity graph through a preset reordering model;

[0048] The response plan formulation module matches the corresponding hazard response plan in the preset safety operation knowledge base according to the target entity chain to carry out safety management and control in the industrial field.

[0049] Compared with the prior art, this application has the following beneficial effects:

[0050] This application integrates multimodal data from the industrial production process to construct a corresponding entity graph, identifies hazardous factors, and screens entity chains in the entity graph by analyzing the dynamic correlation between hazardous factors and entities. Based on the screened target entity chains, corresponding response plans are formulated, thereby improving the accuracy of hazard identification in the industrial production process, screening target entity chains that can reflect the deep correlation between entities, and improving the accuracy of hazard source identification and the efficiency of control measures generation, thereby providing a more scientific and efficient solution for safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a workflow diagram of an industrial field safety management and control method combined with a multimodal model in Example 1 of the present invention;

[0052] Figure 2 Schematic diagram of the entity map in Example 1 of the present invention;

[0053] Figure 3 Schematic diagram of target entity chain screening in Example 1 of the present invention;

[0054] Figure 4 This is a schematic diagram of the structure of an industrial field safety management and control system combined with a multimodal model in Example 2 of the present invention;

[0055] Figure 5 This is a workflow diagram of the hazard source identification method in Example 3 of the present invention;

[0056] Figure 6 This is a workflow diagram of the method for formulating a hidden danger warning plan in Example 3 of the present invention;

[0057] Figure 7 This is a workflow diagram of the hidden danger rectification method in Example 3 of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0060] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0061] Example 1

[0062] This embodiment provides an industrial field security management method combining multimodal models, such as Figure 1 As shown, the industrial field safety management and control method combined with a multimodal model includes:

[0063] S101. Acquire multimodal data from industrial production processes;

[0064] S102: extracting industrial equipment, risks, and operating procedures from the multimodal data as keyword entities, and constructing an entity graph by analyzing the association relationships between the keyword entities;

[0065] S103. Identify hazardous factors in the production process using a preset hazard identification model based on the multimodal data;

[0066] S104: Filtering target entity chains matching the risk factors in the entity graph using a preset reordering model;

[0067] S105. According to the target entity chain, a corresponding risk response plan is matched in a preset safety operation knowledge base to perform safety management and control in the industrial field.

[0068] This embodiment integrates multimodal data from the industrial production process, extracts corresponding keyword entities, establishes connections between corresponding entities by analyzing the association relationships between keyword entities, constructs an entity graph, identifies hazardous factors in the production process, and constructs an association matrix based on the association relationship between hazardous factors and entity graphs, and screens out corresponding target entity chains, and formulates corresponding hazard response plans based on the target entity chains. Compared with traditional methods of managing industrial safety based on single-modal data, this application can comprehensively and timely discover hazardous factors in the production process through multimodal data acquisition and hazard identification models, and find the root causes of hazards through entity graphs and reordering models, and finally match appropriate hazard response plans, effectively reducing the probability of accidents and ensuring the safety of personnel and equipment.

[0069] In this embodiment, multimodal data from industrial production processes, including text, images, videos, and sensor data, is first acquired. Combining these multimodal data for analysis can comprehensively and accurately reflect the actual state of the industrial production process, avoiding the incomplete information problem inherent in single-type data. Industrial equipment, risks, and operating procedures are extracted from the multimodal data as keyword entities. By analyzing the relationships between these keyword entities, an entity graph is constructed. For example, for textual record data, natural language processing techniques are used to identify keyword entities such as industrial equipment names (e.g., "stamping machine," "welding robot"), risk descriptions (e.g., "high temperature burn risk," "mechanical failure risk"), and operating procedures (e.g., "check the mold before starting the stamping machine," "turn off the power after welding"). For image and video data, computer vision techniques are used for image recognition to obtain the corresponding keyword entities. Based on the extracted keyword entities, the relationships between the operating objects and operating procedures are analyzed, and corresponding entities with associated relationships are connected to construct an entity graph. By constructing this entity graph, the overall structure and operational logic of the production system can be quickly understood, enabling targeted measures to be implemented to improve production safety and efficiency.

[0070] At the same time, based on multimodal data, the dangerous conditions in the production process are identified through a preset hazard identification model to obtain the dangerous factors in the production process. The hazard identification model can be a convolutional neural network (CNN), a recurrent neural network (RNN) and other models. By learning various data features when hazards occur in historical data, potential hazards are accurately identified in new production data to obtain dangerous factors. For example, the image is input into the CNN-based hazard identification model, and the model determines whether there are dangerous conditions such as workers' illegal operations and equipment abnormalities in the image. By identifying dangerous factors, potential safety hazards can be discovered in a timely manner, and timely warnings can be issued before the danger occurs and corresponding response measures can be taken to effectively reduce the probability of accidents and ensure production safety.

[0071] Furthermore, through the preset reordering model, the target entity chain matching the hazardous factor is screened out in the entity graph, the degree of association between the entities in the entity graph is calculated, the corresponding paths in the entity graph are sorted, and the target entity chain that best reflects the relevant information of the hazardous factor is screened out; by combining the entity association relationship and the reordering model to screen out the corresponding target entity chain, it is possible to quickly and accurately find the entity chain most relevant to the hazardous factor from a complex entity graph, avoid the interference of irrelevant information, and improve the efficiency and accuracy of decision-making. According to the screened target entity chain, the corresponding hazard response plan is matched in the preset safety operation knowledge base, and professional knowledge, industry standards, internal enterprise safety operation procedures, historical accident handling experience and other information in the field of industrial production are collected and organized into corresponding hazard response measures and stored in the safety operation knowledge base. The target entity chain is used as a retrieval condition to search in the safety operation knowledge base to find the matching hazard response plan. The staff performs the corresponding operations according to the hazard response plan to conduct safety management and control of the industrial production process; by matching the hazard response plan in the safety operation knowledge base, scientific and standardized handling methods can be provided to the staff to avoid the expansion of hazards due to human misjudgment or improper handling. At the same time, the use of existing solutions in the knowledge base can improve processing efficiency, quickly and effectively respond to dangerous situations, ensure the safe and stable operation of industrial production, and reduce losses caused by accidents.

[0072] This application integrates multimodal data from the industrial production process to construct a corresponding entity graph, identifies hazardous factors, and screens entity chains in the entity graph by analyzing the dynamic correlation between hazardous factors and entities. Based on the screened target entity chains, corresponding response plans are formulated, thereby improving the accuracy of hazard identification in the industrial production process, screening target entity chains that can reflect the deep correlation between entities, and improving the accuracy of hazard source identification and the efficiency of control measures generation, thereby providing a more scientific and efficient solution for safety management.

[0073] Furthermore, industrial equipment, risks, and operating steps are extracted from the multimodal data as keyword entities, and an entity graph is constructed by analyzing the association relationship between the keyword entities, including:

[0074] S201: Input the multimodal data into a preset keyword extraction model to extract industrial equipment, risks, and operating steps as keyword entities;

[0075] S202, dividing the keyword entities into different levels according to industrial equipment, risks, and operating steps to obtain multi-level entities;

[0076] S203. Construct an entity graph by analyzing the association relationship between entities in each layer.

[0077] In this embodiment, each modal data in the multimodal data is input into the corresponding preset keyword extraction model respectively. The model extracts the corresponding keyword entity according to the mapping relationship between the data features and the keyword entity. The preset keyword extraction model can be a deep learning model such as a convolutional neural network model and a recurrent neural network. The model is trained with a large amount of historical data to obtain a pre-trained corresponding model. The corresponding modal data is input into the corresponding pre-trained model respectively. For example, keywords such as "stamping machine", "high temperature burn risk", and "check the mold before starting the stamping machine" are marked in a large amount of historical text data; the appearance of the equipment is marked in a large amount of historical image data. , illegal operation scenarios, etc., use a convolutional neural network model, train the model with a large amount of labeled data, adjust the model parameters, enable the model to learn the correspondence between data features and keyword entities, complete the training of the keyword extraction model of the corresponding modality, and obtain a pre-trained keyword extraction model; input the preprocessed multimodal data into the corresponding pre-trained keyword extraction model, and the model extracts the corresponding keyword entities, and extracts keyword entities related to industrial equipment, risks and operation steps respectively; by extracting keyword entities, key information in the industrial production process is obtained, and it is applicable to a variety of complex industrial production scenarios.

[0078] For example, in a machining workshop scenario, text data such as workshop production logs, equipment maintenance records, machine tool operation monitoring video data, and data collected by machine tool vibration and temperature sensors are input into the corresponding trained keyword extraction model. The model extracts keyword entities such as "lathe", "tool wear risk", and "turn off the power before changing the tool" from the text data; identifies the worker's illegal operation of not wearing protective glasses from the video data, and extracts keyword entities such as "risk of not wearing protective glasses"; detects abnormal machine tool vibration from the sensor data, and extracts keyword entities such as "lathe" and "abnormal vibration risk".

[0079] At the same time, based on the extracted keyword entities, the keyword entities are classified according to three categories: industrial equipment, risks and operating steps, and the corresponding keyword entities are placed in the corresponding levels to obtain multi-layer entities. This embodiment is divided into physical layer, logical layer and operation layer, among which the physical layer is industrial equipment, including various machines and devices involved in the production process; the logical layer is risk keywords, including various potential dangerous situations related to equipment operation, personnel operation, etc.; the operation layer is operation steps, which are specific operation behaviors for equipment operation and risk prevention; for example, "injection molding machine", "stamping machine", etc. are divided into physical layer; "mold damage risk", "high temperature burn risk", etc. are divided into logical layer; "check mold temperature before injection molding", "check mold before starting stamping machine", etc. are divided into operation layer; by layering keyword entities, different types of information can be quickly located. When conducting risk analysis or formulating operating specifications, analysis of entities at different levels can quickly find relevant equipment, risks and operating step information, thereby improving information retrieval and analysis efficiency.

[0080] Furthermore, based on the entities at different levels after division, by analyzing the association relationships between the hierarchical entities, the entities with association relationships are connected to construct an entity map. For example, there is a potential dangerous relationship between the physical layer entities and the logical layer entities, and a preventive response relationship between the logical layer entities and the operational layer entities. By analyzing the association relationships between the entities between the layers, connections are established between the corresponding entities to construct the corresponding entity map. By constructing the entity map, the overall operating status of the industrial production process can be controlled, and corresponding preventive measures can be quickly screened out according to the dangerous conditions, thereby improving the scientific nature of production management and the accuracy of decision-making, effectively reducing production risks, and improving production efficiency.

[0081] Furthermore, the entity graph is constructed by analyzing the association relationship between entities in each layer, wherein the multi-layer entities include the physical layer, the logical layer and the operational layer, including:

[0082] S301, obtaining risk association by analyzing the operating status of physical layer entities and the risk triggering probability between logical layer entities;

[0083] S302: Analyze the risk response relationship between the logical layer entity and the operational layer entity to obtain a response association;

[0084] S303: Establish an association between the corresponding physical layer entity and the logical layer entity according to the risk association, and establish an association between the corresponding logical layer entity and the operation layer entity according to the response association to obtain an entity graph.

[0085] In this embodiment, the association relationship between physical layer entities and logical layer entities is analyzed. By analyzing the operating status of the physical layer entities and the risk triggering probability between the logical layer entities, a risk association is obtained. A pre-trained random forest model is used to calculate the corresponding risk triggering probability. A large amount of historical production data, including records of various risks occurring in equipment under different operating states, is used to train the random forest model to obtain a pre-trained random forest model. Based on the input operating status data of the physical layer entities and the learned mapping relationship between the physical layer entities and the logical layer entities, the model calculates the probability value of triggering each risk in the logical layer under the current operating state. Associations are established between entities with probability values ​​greater than a preset threshold (in this embodiment, a probability value greater than 0.5 is considered to be highly correlated). By quantifying the relationship between the operating status of the physical layer entities and the logical layer risks, risks that may be caused by the equipment can be predicted in advance and prevented.

[0086] Furthermore, the association relationship between the logical layer entities and the operational layer entities is analyzed. By analyzing the response relationship between different risks in the logical layer entities and the operational layer entities, the corresponding response association is determined. Based on the professional knowledge in the industrial production field, industry standard documents, internal enterprise safety production regulations and systems, and historical accident handling experience, response measures and operation steps for different risks are extracted to form a risk response knowledge base. Each risk entity in the logical layer is compared with the content in the risk response knowledge base to find all effective response measures and operation steps corresponding to each risk, namely the operational layer entities. The correspondence between the logical layer entities and the operational layer entities is marked to determine which specific operational steps need to be taken to respond to each risk; for example, for "high temperature burn risk", operational layer entities such as "wearing heat-insulating gloves" and "setting high temperature warning signs" are marked as response measures; by establishing associations between corresponding entities, by clarifying the relationship between logical layer risks and operational layer response measures, the corresponding response measures can be quickly searched when risks occur, avoiding the expansion of risks or improper handling due to unclear response measures.

[0087] like Figure 2 As shown, based on the analyzed risk associations, associations are established between the corresponding physical layer entities and logical layer entities. Based on the analyzed response associations, associations are established between the corresponding logical layer entities and operational layer entities to obtain an entity map. By constructing an entity map, we can fully understand the risks faced in the industrial production process and the corresponding preventive measures, reasonably arrange production plans and resource allocation, improve the accuracy and safety of operations, and effectively improve production control in the industrial production process.

[0088] Furthermore, the identifying of hazardous factors in the production process based on the multimodal data using a preset hazard identification model includes:

[0089] S401, extracting features of corresponding modal data from multimodal data respectively, and constructing a joint feature vector;

[0090] S402: Based on the joint feature vector, a preset hazard identification model is used to identify hazardous conditions in the production process to obtain hazardous factors.

[0091] In this embodiment, first, based on the multimodal data in the production process, the features of the corresponding modal data are extracted from the preprocessed multimodal data, and these features are combined into a joint feature vector; for example, for text data, this embodiment adopts a bag-of-words model, and uses a large amount of preprocessed text data to train the bag-of-words model to obtain a pretrained bag-of-words model, and the corresponding text features are extracted through the pretrained bag-of-words model; for image data, this embodiment adopts a convolutional neural network model, and uses a large amount of image data to train the convolutional neural network model to obtain a pretrained convolutional neural network model, and the corresponding image features are extracted through the pretrained convolutional neural network model; the extracted modal features are combined in sequence to obtain a joint feature vector; by constructing the joint feature vector, the complementarity of multimodal data can be fully utilized to improve the accuracy and comprehensiveness of hazardous factor identification; data of different modalities can reflect the situation in the production process from different angles, for example, text data can provide information such as operating instructions and fault records, and image data can intuitively display equipment status and personnel operating behavior. By integrating the features of multimodal data, richer and more comprehensive features can be obtained, thereby more accurately identifying potential dangerous factors and reducing missed detections and false detections.

[0092] Furthermore, based on the fused joint feature vector, a preset hazard identification model is used to identify hazardous conditions in the production process and obtain hazardous factors. The hazard identification model can be a support vector machine, random forest, gradient boosting tree, or other models. In this embodiment, the hazard identification model is specifically a random forest model. The random forest model is trained using joint feature vectors extracted from a large amount of historical multimodal data to obtain a pre-trained random forest model. The fused joint feature vector is then input into the pre-trained random forest model, which outputs a prediction of the hazardous condition. Based on the model output, hazardous factors in the production process are determined, such as equipment failure, illegal personnel operation, and environmental anomalies. By automatically identifying hazardous conditions in industrial production processes, potential hazardous factors can be discovered in a timely manner. Using the hazard identification model for identification can greatly improve identification efficiency and accuracy.

[0093] Furthermore, the method of screening out a target entity chain matching the risk factor in the entity graph by using a preset reordering model includes:

[0094] S501, constructing a correlation matrix by analyzing the correlation between the risk factors and each entity in the entity map;

[0095] S502: Filter entities in each layer of the entity graph using a preset reordering model according to the association matrix, and connect the filtered entities in each layer to obtain a target entity chain.

[0096] In this embodiment, the association relationship between the hazardous factors and each entity in the entity graph is analyzed, and an association matrix is ​​constructed according to the corresponding association degree, and the association degree is used to represent the degree of association between the hazardous factors and each entity. Specifically, each entity in the entity graph is traversed according to the hazardous factors, and the association relationship between each entity and the hazardous factors is analyzed in turn. The entities in the entity graph are used as rows and the hazardous factors are used as columns. According to the association degree obtained by the analysis, the corresponding positions are filled to obtain the association matrix. By constructing the association matrix, the association between the hazardous factors and each entity can be intuitively displayed. Compared with the traditional method of directly matching the corresponding countermeasures according to the hazardous factors, this solution combines the association relationship between the hazardous factors and each entity to screen the countermeasures, which can quickly screen out entities closely related to the hazardous factors, thereby improving the accuracy and efficiency of the analysis.

[0097] Specifically, entities are screened based on the constructed association matrix, and by analyzing the degree of association between the hazardous factors and each entity in the association matrix, entities with a high degree of association with the hazardous factors are screened in each layer of the entity map. A preset reordering model is used to screen out target entity chains that can reflect the causes of hazardous factors, related risks, and response measures. By screening out the target entity chains through the reordering model, entities closely related to hazardous factors can be quickly located from the entity map, and targeted solutions can be formulated, thereby improving the speed of hazardous response and reducing the impact and loss of accidents. This embodiment combines the association matrix and the reordering model to screen the target entity chain, which can accurately find the entity chain closely related to the hazardous factors from the entity map, quickly locate the source of the hazard, and save time and resources.

[0098] Furthermore, the association matrix is ​​constructed by analyzing the association relationship between the risk factors and each entity in the entity map, including:

[0099] S601, extracting corresponding features from the risk factors and constructing a first feature vector;

[0100] S602, mapping each entity into a second feature vector according to the state of each entity in the entity graph;

[0101] S603: Obtain the correlation between the risk factor and each entity by calculating the similarity between the first feature vector and the second feature vector;

[0102] S604, constructing an initial correlation matrix according to the correlation degree;

[0103] S605: Within a preset time period, analyze the hazardous factors and the changes in the industrial production process environment of the entity respectively, update the initial correlation matrix, and obtain a correlation matrix.

[0104] In this embodiment, first, corresponding features are extracted from the hazardous factors. Hazardous factors contain key information related to potential risks. Different types of hazardous factors have different characteristic manifestations. For example, equipment failure hazardous factors are reflected in abnormal changes in equipment operating parameters, while personnel violation hazardous factors are related to personnel behavior and actions. These features are extracted and a first feature vector is constructed. The current hazardous factor is determined to which type it belongs, for example, equipment failure, personnel violation, or environmental abnormality. Based on the type of hazardous factor, feature extraction is performed on different types of hazardous factors. For the image information of the equipment in the equipment failure hazardous factor, a convolutional neural network can be used to extract the visual features of the image. For the personnel violation hazardous factor, the video data is analyzed and the human posture recognition algorithm is used to extract the human movement features, such as limb position and movement amplitude. The extracted features are sequentially integrated to obtain a first feature vector. For example, for an equipment failure hazardous factor composed of equipment temperature, vibration data, and image features, the temperature mean, vibration variance, and image feature vector are sorted to obtain the first feature vector. By constructing the first feature vector, data support is provided for determining the association between the hazardous factor and the entity.

[0105] At the same time, according to the status of each entity in the entity graph, each entity is mapped to a second eigenvector. The entities in the entity graph have different attributes and status information. These status information of the entity are mapped into the same data form as the first eigenvector to obtain the second eigenvector. For each entity in the entity graph, the status information corresponding to the entity is determined. For industrial equipment entities at the physical layer, the status information includes the operating parameters of the equipment, the service life of the equipment, maintenance records, etc.; for risk entities at the logical layer, the status information includes the historical frequency of risk occurrence, the severity level of the risk, etc.; for operation step entities at the operation layer, the status information includes the execution frequency of the step, the execution success rate, etc.; according to the type of entity status information, it is mapped into a vector form. For numerical status information, the numerical value is directly used as the element of the vector; for non-numerical information For information, such as text descriptions in equipment maintenance records, text vectorization methods can be used to convert them into numerical vectors; for classified information, such as the severity level of risk (high, medium, low), one-hot encoding can be used to convert them into vectors; the mapped vectors of various status information of each entity are integrated to obtain the second eigenvector corresponding to the entity; for example, for an industrial equipment entity, its operating parameter vector, service life value, maintenance record text vector, etc. are connected in sequence to obtain the second eigenvector of the equipment entity; by mapping the entity to the second eigenvector, the various status information of the entity can be fully reflected, and when calculating the correlation with the hazardous factor, the intrinsic connection between the entity and the hazardous factor can be more accurately reflected, avoiding analysis errors caused by inconsistent information representation, and improving the accuracy and reliability of the association matrix construction.

[0106] Specifically, the similarity between the first eigenvector and the second eigenvector is calculated to obtain the correlation between the dangerous factor and each entity. The similarity calculation methods include Euclidean distance, cosine similarity, Manhattan distance, etc. In this embodiment, the cosine similarity between the first eigenvector and the second eigenvector is calculated to obtain the correlation between the dangerous factor and each entity. Through the similarity calculation, the degree of correlation between the dangerous factor and the entity can be measured. The higher the similarity, the greater the correlation between the dangerous factor and the entity; according to the calculated correlation, the initial correlation matrix is ​​constructed, the rows of the matrix represent the entities in the entity graph, and the columns represent the dangerous factors. The element values ​​in the matrix are the correlation between the corresponding entities and the dangerous factors.

[0107] Specifically, within a preset time period, the initial association matrix is ​​updated according to the status of hazardous factors and entities and changes in the industrial production process environment. Considering that the status of hazardous factors and entities in the industrial production process will change over time, such as equipment wear, production process adjustments, personnel changes, etc., which will affect the association relationship between hazardous factors and entities, a time period is set according to the actual situation and needs of industrial production, such as every hour, every day, etc. Within the time period, the information related to the hazardous factors and the status information of the entities in the entity map are monitored in real time; according to the monitored environmental changes, the first eigenvector of the hazardous factor and the second eigenvector of the entity are updated, and the corresponding correlation degree is updated, and the initial association matrix is ​​updated using the updated correlation degree to obtain the latest association matrix; by regularly updating the association matrix, the association matrix can adapt to the dynamic changes in the industrial production process, accurately reflect the real-time association relationship between hazardous factors and entities, avoid incorrect association relationship judgment due to changes in the production environment, improve the company's ability to cope with risks, and ensure the safe and stable operation of the production process.

[0108] Furthermore, according to the association matrix, entities are screened in each layer of the entity graph by using a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain, including:

[0109] S701, extracting the correlation degree corresponding to each layer of entities in the entity graph from the correlation matrix, and obtaining the correlation submatrix of each layer of entities;

[0110] S702: Filter out entities with a correlation degree greater than a preset correlation degree threshold according to the correlation submatrix as a candidate entity set for each layer;

[0111] S703: Based on the connection relationship between entities in the entity graph, a plurality of connected entity chains are screened out from the candidate entity set to obtain a candidate entity chain set;

[0112] S704: Analyze the association degree of each entity link in the candidate entity link set using a preset re-ranking model, calculate a re-ranking score, and select the entity link with the highest re-ranking score as the target entity link.

[0113] In this embodiment, the correlation degree corresponding to each layer of entities is extracted from the correlation matrix, and a correlation sub-matrix is ​​constructed. The overall correlation matrix is ​​decomposed by level, and each layer of entities is screened respectively. By dividing the correlation matrix into correlation sub-matrices, hierarchical screening is achieved. The entities at different levels have different association modes and influence degrees with the risk factors. The hierarchical processing can more accurately grasp the characteristics of each layer of entities. According to the correlation sub-matrix, entities that are closely associated with the risk factors and play an important role in the generation or response of the risk are screened from each correlation sub-matrix respectively. By selecting entities with a correlation degree greater than a preset correlation degree threshold, the entities at each level are selected as the risk factors. The candidate entity set of the layer is set according to the actual production scenario and historical data analysis, and the correlation threshold is set; for example, through the statistical analysis of the correlation between entities and hazardous factors in historical hazardous events, it is found that entities with a correlation higher than 0.6 play a key role in the occurrence and treatment of hazards, and the correlation threshold is set to 0.6; each correlation sub-matrix is ​​traversed, and the entities corresponding to the elements with a correlation greater than 0.6 are screened out to obtain the corresponding candidate entity set; by screening the candidate entities, entities that have little to do with hazardous factors can be quickly eliminated, the screening speed can be accelerated, and the accuracy of the judgment of entities related to hazardous factors can be improved.

[0114] Specifically, in the screened candidate entity set, multiple entity chains are obtained according to the connection relationship between candidate entities at different levels, and a candidate entity chain set is constructed. Starting from the physical layer candidate entity, the logical layer candidate entity connected to it is found, and then the corresponding operation layer candidate entity is further found to form a complete entity chain. All qualified entity chains are summarized to obtain a candidate entity chain set; by screening the candidate entity chain set, the causal relationship and response path related to the hazardous factors are displayed in the form of a chain.

[0115] like Figure 3As shown, the entity chain with the closest relationship with the dangerous factor is screened out from the candidate entity chain set as the target entity chain, and the degree of association between each candidate entity chain and the dangerous factor is calculated by the preset re-ranking model to obtain a corresponding score. The higher the score, the closer the relationship between the entity chain and the dangerous factor, and the more critical it is to the generation and treatment of the danger. The entity chain with the highest score is screened out as the target entity chain; the re-ranking model can be a sorting algorithm based on gradient boosting, a neural network sorting model, etc. The re-ranking model of this embodiment is specifically a neural network sorting model, which uses a large amount of historical dangerous event data, including dangerous factors, corresponding entity maps, generated candidate entity chain sets, and the actual processing process. The neural network sorting model is trained based on the target entity chain to obtain a pre-trained re-ranking model. Each entity chain in the candidate entity chain set is input into the pre-trained re-ranking model. The model calculates the re-ranking score of each entity chain based on factors such as the sum of the association degrees of each entity in the entity chain, the length of the chain, and the tightness of the inter-layer connection. The entity chain with the highest re-ranking score is selected as the target entity chain in the candidate entity chain set. By screening out the target entity chain through the re-ranking model, the entity chain with the closest and most critical association with the hazardous factors can be accurately found. The target entity chain reflects the root cause of the hazard, the risks involved, and the effective response measures, which can improve the efficiency of risk handling and reduce accident losses.

[0116] Furthermore, the target entity chain is matched with corresponding hazard response plans in a preset safety operation knowledge base to perform safety management and control in the industrial field, including:

[0117] S801. According to the target entity chain, a match is performed in a preset security operation knowledge base in hierarchical order to obtain corresponding response measures;

[0118] S802: Dynamically combine the response measures based on the correlation between entities in the target entity chain to obtain corresponding hazard response plans to perform safety management and control in the industrial field.

[0119] In this embodiment, according to the filtered target entity chain, the preset safety operation knowledge base is matched according to the hierarchical order of the target entity chain. The preset safety operation knowledge base stores the response measures for various industrial production risks. According to the hierarchical division of the entity map, starting from the physical layer, the response measures related to the physical layer entity are searched in the safety operation knowledge base; then the response measures corresponding to the logical layer entity are searched; finally, the response measures of the operation layer entity are searched; for example, for the equipment entity of the physical layer, the relevant measures under the keywords such as "equipment maintenance" and "equipment inspection" are searched in the knowledge base; for the risk entity of the logical layer, the relevant measures under the keywords such as "risk prevention" and "risk prevention" are searched. Measures under keywords such as "risk warning"; according to the hierarchical matching method, matching is carried out in the safety operation knowledge base in turn, using the entities in the target entity chain as keywords, and searching for matching or semantically similar response measures under the corresponding categories of the knowledge base; matching response measures in the knowledge base in hierarchical order can avoid blind search and improve matching efficiency. At the same time, the hierarchical matching method can ensure the correspondence between response measures and entities at each level in the target entity chain, making the acquired response measures more targeted, and being able to comprehensively cover the links involved in hazardous factors from multiple levels such as equipment operation, risk prevention, and operation execution, thereby formulating effective hazard response plans.

[0120] Specifically, the matched response measures are dynamically combined. Through the correlation between entities in the target entity chain, the importance and mutual influence of entities in the process of hazard generation and development are analyzed. These measures are dynamically combined according to the correlation between entities. This can highlight the response strategies of key links and weaken the measures of secondary links, making the hazard response plan more in line with the actual hazard situation and enhancing the pertinence and effectiveness of the plan.

[0121] Furthermore, the response measures are dynamically combined based on the association between entities in the target entity chain to obtain corresponding hazard response plans for safety management and control in the industrial field, including:

[0122] S901. According to the target entity chain, extract the total correlation between the corresponding entity and the connected entities from the correlation matrix;

[0123] S902: assigning a corresponding priority weight to each entity in the target entity chain according to the total relevance;

[0124] S903. Dynamically combine corresponding response measures according to the priority weights to obtain corresponding hazard response plans to perform safety management and control in the industrial field.

[0125] In this embodiment, based on the target entity chain, the total correlation between each entity in the target entity chain and the connected entities is extracted in the association matrix to quantify the importance and influence of each entity in the chain. The higher the total correlation, the more critical the role of the entity in the generation, development or response of the risk. For each entity in the target entity chain, the entity directly connected to it in the entity graph is found, and the corresponding correlation value is extracted from the association matrix. The correlation values ​​of each entity and the connected entities are summed to obtain the total correlation of the entity. By extracting the total correlation, the importance of each entity in the target entity chain can be reflected, which helps to accurately grasp the key links of the risk and improve the pertinence and effectiveness of the risk response.

[0126] Specifically, based on the calculated total correlation, a corresponding priority weight is assigned to each entity in the target entity chain. The higher the priority weight, the more important the response measures corresponding to the entity are in the risk response plan, and they need to be executed first or focused on. By assigning priority weights, when formulating risk response plans, the execution order and resource investment of response measures can be formulated to ensure that key issues are resolved first. This embodiment uses a normalization method to convert the total correlation into a priority weight value between 0 and 1, and calculates the priority weight of each entity in the target entity chain. By assigning priority weights, the importance order of each entity in the target entity chain can be determined, making the formulation of risk response plans more logical and sequential, giving priority to key issues, and improving resource utilization efficiency.

[0127] Specifically, after calculating the priority weight of each entity, the response measures are integrated according to the weight value to obtain a complete, orderly and targeted hazard response plan. The response measures with high weight are executed first to ensure that the hazard is effectively controlled. The response measures corresponding to each entity are sorted according to the priority weight of the corresponding entity. The response measures corresponding to the entity with high weight are ranked in front. According to the sorting result, the response measures are combined to obtain a preliminary hazard response plan; for example, the high-weight response measures corresponding to "overpressure risk" are listed first, and then the measures corresponding to "reactor" and "emergency pressure relief operation" are listed in turn; the preliminary combined hazard response plan is optimized to determine the execution time and execution time of each response measure Details such as personnel and execution conditions are reviewed. At the same time, the logic and feasibility of the plan are checked to ensure that each measure cooperates and works together to obtain the final hazard response plan. For example, it is stipulated that "installation of pressure alarm device" must be completed within 24 hours after the start of the plan, and the equipment maintenance department is responsible for it; "regular inspection of reactor sealing" is carried out once a week and is executed by the production team, etc. Through the dynamic combination of response measures with priority weights, the hazard response plan is closely centered around the key links of the hazard, which improves the pertinence and effectiveness of the plan and reduces the possibility and degree of harm of accidents. At the same time, specific plans are helpful for operators to understand and implement, reduce communication costs and operational errors, and enhance the company's overall safety management and control capabilities and emergency response levels.

[0128] Example 2

[0129] In this embodiment, if Figure 4 , provides an industrial field safety management and control system combining a multimodal model, for implementing the industrial field safety management and control method combining a multimodal model, including:

[0130] Data acquisition module, which acquires multimodal data from industrial production processes;

[0131] An entity graph construction module extracts industrial equipment, risks, and operating steps from the multimodal data as keyword entities, and constructs an entity graph by analyzing the association relationship between the keyword entities;

[0132] A hazardous factor identification module, which identifies hazardous factors in the production process based on the multimodal data and a preset hazardous identification model;

[0133] An entity chain screening module, which screens target entity chains matching the risk factors in the entity graph through a preset reordering model;

[0134] The response plan formulation module matches the corresponding hazard response plan in the preset safety operation knowledge base according to the target entity chain to carry out safety management and control in the industrial field.

[0135] In this embodiment, the data acquisition module comprehensively and in real time collects multimodal data in the industrial production process, and obtains various types of data including text, images, video data, etc. For example, text data such as equipment operation logs and operation records are obtained from the production management system, and image and video data of equipment operation status and personnel operation behavior are collected through workshop cameras. The collected data undergoes preliminary cleaning and preprocessing to remove noise and invalid data, providing data support for industrial safety analysis; the entity map construction module mines key information in the industrial production process based on the multimodal data provided by the data acquisition module, constructs a corresponding network, extracts industrial equipment, risks and operating steps from the multimodal data as keyword entities, and divides the keyword entities into different levels according to industrial equipment, risks and operating steps to form a multi-layer Entity structure, by analyzing the association relationship between entities at each level, constructing an entity map, and displaying the intrinsic connection between various elements in the industrial production process through the entity map, providing a knowledge framework for subsequent risk analysis and response decision-making; the hazardous factor identification module identifies hazardous conditions in the production process based on multimodal data through a preset hazard identification model, extracts corresponding features from the multimodal data, constructs feature vectors, and inputs the feature vectors into the hazard identification model. The model analyzes and judges the current production process by learning the mapping relationship between features and hazardous conditions in historical data, and identifies hazardous factors, including equipment failure, personnel illegal operation, environmental abnormalities and other hazardous conditions. By identifying hazardous factors, safety hazards can be discovered in advance, so that preventive measures can be taken to reduce the probability of accidents.

[0136] Specifically, based on the entity map, the entity chain screening module screens out target entity chains related to hazardous factors, and constructs an association matrix by analyzing the association relationship between hazardous factors and each entity in the entity map to obtain the degree of association between hazardous factors and each entity. Combined with the connection relationship between inter-layer entities in the entity map, the entity chain is screened to obtain the target entity chain with the highest matching degree with the hazardous factors. By screening out the target entity chain, a direction can be provided for the formulation of hazardous response plans; the response plan formulation module matches and generates corresponding hazardous response plans in the preset safety operation knowledge base according to the screened target entity chain, and matches in the safety operation knowledge base according to the hierarchical order of the target entity chain to obtain response measures corresponding to each entity in the target entity chain. Combined with the association between entities in the target entity chain, the matched response measures are dynamically combined to form a complete, orderly and targeted hazardous response plan. Rapid response to hazardous conditions according to the hazardous response plan can reduce accident losses and ensure the safe and stable operation of industrial production.

[0137] Example 3

[0138] In this embodiment, a specific implementation method of an industrial field safety management and control method combining a multimodal model is provided. Figure 5 As shown, first, a small model of a vectorization tool is used to structure the safety management documents such as industry standards and corporate specifications. The tool uses a pre-trained deep learning model to convert text content into high-dimensional vectors and build a "safety management document vector knowledge base". The knowledge base supports dynamic updates to ensure synchronization with the latest regulations and standards.

[0139] When users ask risk-related queries in natural language, the system leverages a large AI multimodal model to achieve semantic understanding and contextual association. The model then searches the "Security Management Document Vector Knowledge Base," filters key information through an attention mechanism, and generates precise responses consistent with industry terminology. The AI ​​multimodal model supports multimodal inputs such as text, images, and structured data, enhancing its adaptability to complex scenarios.

[0140] The system uses the zero-sample or few-sample learning capabilities of large models to analyze equipment parameters, environmental data, historical accident records, etc., and automatically identify major hazards; by comparing the risk feature library in the knowledge base (such as chemical flash point, pressure vessel threshold), it outputs a list of hazards and their potential hazard chains.

[0141] Based on the recognition results, the system calls a preset risk matrix algorithm (such as the LEC method or the MES method), combines the large model with the quantitative assessment of uncertainty factors (such as the probability of human operation errors), generates a dynamic risk level, and introduces an adversarial training mechanism into the assessment process to ensure the correctness of the results in extreme scenarios.

[0142] Through the generative AI capabilities of AI multimodal large models, hierarchical control measures are output, including technical improvements, emergency plans, training requirements, etc. The measures are generated using a reinforcement learning framework, and actual control effect feedback is continuously absorbed for iterative optimization to form closed-loop management.

[0143] like Figure 6 As shown, a small vectorization tool model is used to extract features and vectorize structured text such as "Developed Safety Inspection Plans" and "Hazard Investigation Standards" to populate the "Safety Management Document Vector Knowledge Base." This example uses a lightweight Transformer-based model to achieve highly compressed semantic representation, ensuring high accuracy of the knowledge base content.

[0144] The system uses AI multimodal models to perform cross-modal alignment and feature fusion of multimodal data such as inspection images, videos, and audio data transmitted back in real time by on-site terminals. It adopts a spatiotemporal attention mechanism to analyze the operating status of equipment and personnel in the video. At the same time, it combines voiceprint recognition technology to analyze audio anomalies, such as abnormal mechanical noises, pipe leakage screams, equipment roars, etc., to achieve multi-dimensional data joint modeling.

[0145] The AI ​​multimodal large model compares knowledge base search results with real-time data analysis to output hidden danger location and risk type determination. Monte Carlo Dropout technology is used to calculate confidence probabilities, complete self-scoring of the results, and generate a quantitative assessment report based on relevant national standards. When the confidence level falls below a preset threshold, a manual review process is automatically triggered. Based on the diagnostic results, a graded hidden danger warning plan is output, including emergency response steps, resource scheduling recommendations, and standardized operational instructions, which are simultaneously pushed to mobile devices and the central control platform.

[0146] like Figure 7 As shown, the AI ​​multimodal large model transforms pre-organized hidden danger rectification standards, historical case libraries, and benefit analysis reports into structured vectors using a small vectorization tool model to populate the "Safety Management Document Vector Knowledge Base." This management method employs FAISS technology, or approximate nearest neighbor search, to achieve rapid matching of clauses, specifications, treatment measures, and case studies. For identified hidden dangers, such as text descriptions, on-site images, and sensor data, the AI ​​multimodal large model performs feature fusion analysis. Using visual capabilities, it extracts equipment defect features from the images. Combined with keyword entities in the text descriptions, it searches the "Safety Management Document Vector Knowledge Base" for historical cases and corresponding standard clauses with a similarity above a threshold.

[0147] When the hidden danger risk is triggered by exceeding the threshold, the AI ​​multimodal large model analysis capability is combined with the confirmed hidden danger to generate corrective measures: Level 1 measures: immediate implementation of measures, such as "shutdown and pressure relief" and other emergency measures; Level 2 measures: implementation within 24 hours, such as "ultrasonic thickness detection" and other technical means; Level 3 measures: long-term improvement, such as "anti-corrosion coating upgrade" and other engineering modifications. At the same time, in the safety management method of the present invention, the rectification plan can be automatically associated with the material list through Agent Assistant analysis, and at the same time, work instructions and cost-benefit analysis reports are compiled. Manual verification is carried out on the corrective measures provided by the multimodal AI model, and manual acceptance is carried out after verification, and finally the hidden danger rectification is completed.

[0148] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for industrial field safety management and control combined with a multimodal model, characterized in that: include: Acquire multimodal data from industrial production processes; Extracting industrial equipment, risks, and operating steps from the multimodal data as keyword entities, respectively, and constructing an entity graph by analyzing the association relationships between the keyword entities; Identifying hazardous factors in the production process using a preset hazard identification model based on the multimodal data; Using a preset re-ranking model, a target entity chain matching the risk factor is screened out from the entity graph; According to the target entity chain, the corresponding hazard response plan is matched in the preset safety operation knowledge base to perform safety management and control in the industrial field; The method extracts industrial equipment, risks, and operating steps from the multimodal data as keyword entities, and constructs an entity graph by analyzing the association relationship between the keyword entities, including: Inputting the multimodal data into a preset keyword extraction model respectively, extracting industrial equipment, risks, and operating steps as keyword entities; Divide the keyword entities into different levels according to industrial equipment, risks and operation steps to obtain multi-layer entities; wherein the multi-layer entities include a physical layer, a logical layer and an operation layer; By analyzing the operating status of physical layer entities and the risk triggering probability between logical layer entities, risk association is obtained; By analyzing the risk response relationship between the logical layer entities and the operational layer entities, the response association is obtained; Establishing an association between the corresponding physical layer entity and the logical layer entity according to the risk association, and establishing an association between the corresponding logical layer entity and the operation layer entity according to the response association, to obtain an entity graph; The method of screening out a target entity chain matching the risk factor in the entity graph by using a preset reordering model includes: Extract corresponding features from the dangerous factors and construct the first feature vector; According to the state of each entity in the entity graph, each entity is mapped into a second feature vector; Obtaining the correlation between the risk factor and each entity by calculating the similarity between the first feature vector and the second feature vector; constructing an initial correlation matrix according to the correlation degree; Within a preset time period, the hazardous factors and the changes in the industrial production process environment of the entity are analyzed respectively, and the initial correlation matrix is ​​updated to obtain a correlation matrix; Extract the correlation degree corresponding to each layer of entities in the entity graph from the correlation matrix to obtain the correlation submatrix of each layer of entities; According to the correlation submatrix, entities with a correlation degree greater than a preset correlation degree threshold are screened out as a candidate entity set for each layer; In combination with the connection relationship between entities in the entity graph, a plurality of connected entity chains are screened out from the candidate entity set to obtain a candidate entity chain set; The preset reordering model is used to analyze the sum of entity associations, chain length, and inter-layer connection density of each entity chain in the candidate entity chain set, calculate a reordering score, and select the entity chain with the highest reordering score as the target entity chain.

2. The industrial field safety management and control method combined with a multimodal model according to claim 1 is characterized in that: The step of identifying hazardous factors in the production process based on the multimodal data using a preset hazard identification model includes: From the multimodal data, the features of the corresponding modal data are extracted respectively and the joint feature vector is constructed; According to the joint feature vector, the dangerous conditions in the production process are identified through a preset danger identification model to obtain the dangerous factors.

3. The industrial field safety management and control method combined with a multimodal model according to claim 1 is characterized in that: According to the target entity chain, a corresponding hazard response plan is matched in a preset safety operation knowledge base to perform safety management and control in the industrial field, including: According to the target entity chain, the system matches the preset security operation knowledge base in hierarchical order to obtain the corresponding response measures; Combined with the correlation between entities in the target entity chain, the response measures are dynamically combined to obtain corresponding hazard response plans to conduct safety management and control in the industrial field.

4. The industrial field safety management and control method combined with a multimodal model according to claim 3 is characterized in that: The response measures are dynamically combined based on the correlation between entities in the target entity chain to obtain corresponding hazard response plans to perform safety management and control in the industrial field, including: According to the target entity chain, the total correlation between the corresponding entity and the connected entities is extracted from the correlation matrix; According to the total relevance, assign a corresponding priority weight to each entity in the target entity chain; According to the priority weights, the corresponding response measures are dynamically combined to obtain corresponding hazard response plans to conduct safety management and control in the industrial field.

5. An industrial safety management and control system combining a multimodal model, characterized in that: A method for industrial field safety management and control combined with a multimodal model according to any one of claims 1 to 4 is implemented, comprising: Data acquisition module, which acquires multimodal data from industrial production processes; An entity graph construction module extracts industrial equipment, risks, and operating steps from the multimodal data as keyword entities, and constructs an entity graph by analyzing the association relationship between the keyword entities; A hazardous factor identification module, which identifies hazardous factors in the production process based on the multimodal data and a preset hazardous identification model; An entity chain screening module, which screens target entity chains matching the risk factors in the entity graph through a preset reordering model; The response plan formulation module matches the corresponding hazard response plan in the preset safety operation knowledge base according to the target entity chain to carry out safety management and control in the industrial field.

Images