Industrial field safety management and control method and system combined with multi-modal model

Abstract

The invention relates to the technical field of safety management and control, and discloses an industrial field safety management and control method and system combined with a multi-modal model, and the method comprises the steps: obtaining multi-modal data in an industrial field production process; industrial equipment, risks and operation steps are extracted from the multi-modal data to serve as keyword entities, and an entity map is constructed by analyzing the incidence relation between the keyword entities; according to the multi-modal data, through a preset risk identification model, identifying risk factors in the production process; screening out a target entity chain matched with the risk factor from the entity map through a preset reordering model; and according to the target entity chain, matching a corresponding danger coping scheme in a preset safety operation knowledge base so as to perform safety control on the industrial field, according to the method and the device, the dangerous source identification accuracy and the management and control measure generation efficiency can be improved, so that a more scientific and efficient solution is provided for safety management and control.

Description

Technical Field

[0001] The present invention relates to the technical field of safety control, and particularly to an industrial field safety control method and system combining a multi-modal model. Background Art

[0002] In the industrial field, safety control can reduce the accident rate by monitoring and analyzing data such as equipment operation status, personnel operation behavior, and process parameters in real time, and identifying potential danger information in advance; however, with the improvement of industrial production complexity and automation level, traditional single safety control methods based on single-modal data are difficult to cope with new risk challenges.

[0003] The existing technologies have the following problems: data based on a single modality cannot reflect the overall industrial production status, resulting in a high risk of missed detections in complex scenarios; directly matching by keywords cannot capture the deep semantic association relationships between entities, resulting in poor correlation between retrieved relevant countermeasures; a single method for formulating countermeasures is difficult to cope with the dynamic changes in industrial scenarios, resulting in poor effectiveness of the formulated solutions; to solve at least one of the above problems, the present invention proposes an industrial field safety control method and system combining a multi-modal model. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the main object of the present invention is to provide an industrial field safety control method and system combining a multi-modal model, which can effectively solve the problems in the background art. The specific technical solutions of the present invention are as follows: An industrial field safety control method combining a multi-modal model, comprising: Obtaining multi-modal data in the industrial production process; Respectively extracting industrial equipment, risks, and operation steps from the multi-modal data as keyword entities, and constructing an entity map by analyzing the association relationships between the keyword entities; According to the multi-modal data, identifying dangerous elements in the production process through a preset danger identification model; Screening out target entity chains matching the dangerous elements in the entity map through a preset reordering model; According to the target entity chain, matching corresponding danger countermeasures in a preset safety operation knowledge base to conduct safety control over the industrial field.

[0005] Specifically, the step of respectively extracting industrial equipment, risks, and operation steps from the multi-modal data as keyword entities, and constructing an entity map by analyzing the association relationships between the keyword entities, includes: Input multi-modal data into a preset keyword extraction model respectively, and extract industrial equipment, risks, and operation steps as keyword entities; Divide the keyword entities into different levels according to industrial equipment, risks, and operation steps to obtain multi-level entities; Construct an entity map by analyzing the association relationships between entities at each level.

[0006] Specifically, for constructing the entity map by analyzing the association relationships between entities at each level, where the multi-level entities include a physical layer, a logic layer, and an operation layer, it includes: Obtain risk associations by analyzing the risk triggering probabilities between the operating states of physical layer entities and logic layer entities; Obtain response associations by analyzing the risk response relationships between logic layer entities and operation layer entities; Establish associations between corresponding physical layer entities and logic layer entities according to the risk associations, and establish associations between corresponding logic layer entities and operation layer entities according to the response associations to obtain the entity map.

[0007] Specifically, for identifying dangerous elements in the production process according to the multi-modal data through a preset danger identification model, it includes: Extract the features of the corresponding modal data from the multi-modal data respectively to construct a joint feature vector; Identify the dangerous conditions in the production process through the preset danger identification model according to the joint feature vector to obtain dangerous elements.

[0008] Specifically, for screening out target entity chains matching the dangerous elements in the entity map through a preset re-ranking model, it includes: Construct an association matrix by analyzing the association relationships between the dangerous elements and each entity in the entity map; According to the association matrix, perform entity screening in each layer of the entity map through the preset re-ranking model, and connect the screened entities in each layer to obtain the target entity chain.

[0009] Specifically, for constructing the association matrix by analyzing the association relationships between the dangerous elements and each entity in the entity map, it includes: Extract corresponding features from the dangerous elements to construct a first feature vector; Map each entity to a second feature vector according to the state of each entity in the entity map; Obtain the association degrees between the dangerous elements and each entity by calculating the similarity between the first feature vector and the second feature vector; Construct an initial association matrix according to the association degrees; Within a preset time period, analyze the changes in the industrial production process environment of the risk factors and entities respectively, and update the initial association matrix to obtain an association matrix.

[0010] Specifically, according to the association matrix, entity screening is performed in each layer of the entity graph through a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain, including: Extract the association degrees corresponding to the entities in each layer of the entity graph from the association matrix respectively to obtain an association sub-matrix for each layer of entities; According to the association sub-matrix, screen out the entities with association degrees greater than the preset association degree threshold as the candidate entity set for each layer; Combined with the connection relationship between the entities in different layers of the entity graph, screen out multiple connected entity chains from the candidate entity set to obtain a candidate entity chain set; Through a preset reordering model, analyze the association degree of each entity chain in the candidate entity chain set, calculate a reordering score, and screen out the entity chain with the highest reordering score as the target entity chain.

[0011] Specifically, according to the target entity chain, match the corresponding risk response plan in a preset safety operation knowledge base to conduct safety control over the industrial field, including: According to the target entity chain, perform matching in the preset safety operation knowledge base in hierarchical order to obtain corresponding countermeasures; Combined with the association degree between the entities in the target entity chain, dynamically combine the countermeasures to obtain the corresponding risk response plan for safety control of the industrial field.

[0012] Specifically, the step of combining the association degree between the entities in the target entity chain to dynamically combine the countermeasures to obtain the corresponding risk response plan for safety control of the industrial field includes: According to the target entity chain, extract the total association degree between the corresponding entity and the connected entity from the association matrix; According to the total association degree, assign corresponding priority weights to each entity in the target entity chain; According to the priority weights, dynamically combine the corresponding countermeasures to obtain the corresponding risk response plan for safety control of the industrial field.

[0013] An industrial field safety control system combining a multi-modal model for implementing the industrial field safety control method combining a multi-modal model includes: A data acquisition module for acquiring multi-modal data in the industrial production process; The entity graph construction module extracts industrial equipment, risks, and operation steps from the multimodal data as keyword entities respectively, and constructs an entity graph by analyzing the association relationships between the keyword entities; The hazard factor identification module identifies the hazard factors in the production process according to the multimodal data through a preset hazard identification model; The entity chain screening module screens out the target entity chains matching the hazard factors in the entity graph through a preset re-ranking model; The countermeasure formulation module matches the corresponding hazard countermeasures in a preset safety operation knowledge base according to the target entity chains to conduct safety control over the industrial field.

[0014] Compared with the prior art, the present application has the following beneficial effects: The present application constructs a corresponding entity graph by integrating multimodal data in the industrial production process, identifies hazard factors, analyzes the dynamic correlation degree between hazard factors and entities, screens entity chains in the entity graph, and formulates corresponding countermeasures based on the screened target entity chains, improving the accuracy of hazard identification in the industrial production process, screening out target entity chains that can reflect the deep associations between entities, improving the accuracy rate of hazard source identification and the efficiency of generating control measures, and thus providing a more scientific and efficient solution for safety management. Description of the Drawings

[0015] Figure 1 It is a working flowchart of a safety control method for the industrial field combining a multimodal model in Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the entity graph in Embodiment 1 of the present invention; Figure 3 It is a schematic diagram of screening the target entity chain in Embodiment 1 of the present invention; Figure 4 It is a schematic structural diagram of a safety control system for the industrial field combining a multimodal model in Embodiment 2 of the present invention; Figure 5 It is a working flowchart of a hazard source identification method in Embodiment 3 of the present invention; Figure 6 It is a working flowchart of a hidden danger warning plan formulation method in Embodiment 3 of the present invention; Figure 7 It is a working flowchart of a hidden danger rectification method in Embodiment 3 of the present invention. Detailed Embodiments

[0016] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be made in conjunction with the accompanying drawings of the specification.

[0017] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways different from those described herein. Persons skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0018] Secondly, as used herein, an "embodiment" or "embodiments" refers to specific features, structures, or characteristics that may be included in at least one implementation manner of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it an individual or alternative embodiment that excludes other embodiments.

[0019] Embodiment 1

[0020] This embodiment provides a safety control method for the industrial field in combination with a multimodal model. As Figure 1 shown, the safety control method for the industrial field in combination with a multimodal model includes: S101. Obtain multimodal data in the production process of the industrial field; S102. Respectively extract industrial equipment, risks, and operation steps from the multimodal data as keyword entities, and construct an entity map by analyzing the association relationships between the keyword entities; S103. According to the multimodal data, identify dangerous elements in the production process through a preset danger recognition model; S104. Through a preset reordering model, screen out target entity chains matching the dangerous elements in the entity map; S105. According to the target entity chain, match corresponding danger response plans in a preset safety operation knowledge base to perform safety control on the industrial field.

[0021] This embodiment fuses the multimodal data in the production process of the industrial field, extracts the corresponding keyword entities, establishes connections between the corresponding entities by analyzing the association relationships between the keyword entities, constructs an entity map, identifies the dangerous elements in the production process, combines the association relationships between the dangerous elements and the entity map, constructs an association matrix and screens out the corresponding target entity chains, and formulates corresponding danger response plans according to the target entity chains. Compared with the traditional method of managing industrial field safety based on single-modal data, this application can comprehensively and timely discover the dangerous elements in the production process through multimodal data acquisition and a danger recognition model, find the root cause of the danger through the entity map and the reordering model, and finally match the appropriate danger response plan, effectively reducing the probability of accidents and ensuring the safety of personnel and equipment.

[0022] In this embodiment, first, multi-modal data in the industrial production process is obtained, including data such as text, images, videos, and sensor data. Analyzing the multi-modal data can comprehensively and accurately reflect the actual situation of the industrial production process, avoiding the problem of incomplete information in a single type of data. Industrial equipment, risks, and operation steps are respectively extracted from the multi-modal data as keyword entities, and an entity map is constructed by analyzing the association relationships between the keyword entities. For example, for text record data, keyword entities such as industrial equipment names (such as "stamping machine", "welding robot"), risk descriptions (such as "risk of high-temperature scald", "risk of mechanical failure"), and operation steps (such as "check the mold before starting the stamping machine", "turn off the power after welding") are identified through natural language processing technology. For image and video data, corresponding keyword entities are obtained by using computer vision technology for image recognition. According to the extracted keyword entities, the relationship between the operation object and the operation step is analyzed, and the corresponding entities with an association relationship are connected to construct an entity map. By constructing the entity map, the overall structure and operation logic of the production system can be quickly grasped, and targeted measures can be taken to improve production safety and efficiency.

[0023] At the same time, according to the multi-modal data, the dangerous situation in the production process is identified through a preset danger recognition model to obtain the dangerous elements in the production process. The danger recognition model can be a model such as a convolutional neural network (CNN) or a recurrent neural network (RNN). By learning various data features when a danger occurs in historical data, potential dangers can be accurately identified in new production data to obtain the dangerous elements. For example, when an image is input into the danger recognition model based on CNN, the model determines whether there are dangerous situations such as workers' illegal operations or equipment abnormalities in the image. By identifying the dangerous elements, potential safety hazards can be discovered in a timely manner, and early warnings can be given and corresponding countermeasures can be taken before the danger occurs, effectively reducing the probability of accidents and ensuring production safety.

[0024] Further, through a preset reordering model, target entity chains matching the risk factors are screened out in the entity graph, the degree of association between entities in the entity graph is calculated, the corresponding paths in the entity graph are sorted, and the target entity chain that can best reflect the information related to the risk factors is screened out; by combining the entity association relationship and the reordering model to screen out the corresponding target entity chain, the entity chain most relevant to the risk factors can be quickly and accurately found from the complex entity graph, avoiding the interference of irrelevant information, and improving the efficiency and accuracy of decision-making. According to the screened target entity chain, the corresponding risk response plan is matched in a preset safety operation knowledge base. Information such as professional knowledge in the industrial production field, industry standards, internal safety operation procedures of the enterprise, and historical accident handling experience is collected and sorted into corresponding risk response measures, which are stored in the safety operation knowledge base. Using the target entity chain as the retrieval condition, a search is performed in the safety operation knowledge base to find the matching risk response plan, and the staff performs corresponding operations according to the risk response plan to conduct safety control over the industrial production process; by matching the risk response plan in the safety operation knowledge base, a scientific and standardized processing method can be provided for the staff, avoiding the expansion of risks caused by human judgment errors or improper handling. At the same time, the existing plans in the knowledge base can be used to improve the processing efficiency, quickly and effectively respond to dangerous situations, ensure the safe and stable operation of industrial production, and reduce the losses caused by accidents.

[0025] This application constructs a corresponding entity graph by integrating multi-modal data in the industrial production process, identifies risk factors, analyzes the dynamic association degree between risk factors and entities, screens entity chains in the entity graph, and formulates corresponding response plans based on the screened target entity chains, improving the accuracy of risk identification in the industrial production process, screening out target entity chains that can reflect the deep association between entities, improving the accuracy of hazard source identification and the efficiency of generating control measures, and thus providing a more scientific and efficient solution for safety management.

[0026] Further, industrial equipment, risks, and operation steps are respectively extracted from the multi-modal data as keyword entities, and an entity graph is constructed by analyzing the association relationship between the keyword entities, including: S201. Input the multi-modal data into a preset keyword extraction model respectively, and extract industrial equipment, risks, and operation steps as keyword entities; S202. Divide the keyword entities into different levels according to industrial equipment, risks, and operation steps to obtain multi-level entities; S203. Construct an entity graph by analyzing the association relationship between entities at each level.

[0027] In this embodiment, each modality data in the multimodal data is respectively input into the corresponding preset keyword extraction model. The model extracts the corresponding keyword entities according to the mapping relationship between the data features and the keyword entities. The preset keyword extraction model can be a deep learning model such as a convolutional neural network model or a recurrent neural network. The model is trained with a large amount of historical data to obtain the corresponding pre-trained model. The corresponding modality data is respectively input into the corresponding pre-trained model. For example, keywords such as "stamping machine", "risk of high-temperature scald", and "check the mold before starting the stamping machine" are marked in a large amount of historical text data; corresponding keyword entities such as the appearance of the equipment and the scene of illegal operations are marked in a large amount of historical image data. The convolutional neural network model is used to train the model with the marked large amount of data, and the model parameters are adjusted to enable the model to learn the corresponding relationship between the data features and the keyword entities, and the training of the keyword extraction model for the corresponding modality is respectively completed to obtain the pre-trained keyword extraction model; the preprocessed multimodal data is respectively input into the corresponding pre-trained keyword extraction model, and the model extracts the corresponding keyword entities, and the keyword entities related to industrial equipment, risks, and operation steps are respectively extracted; by extracting the keyword entities, the key information in the industrial production process is obtained, and it is applicable to a variety of complex industrial production scenarios.

[0028] Exemplarily, in the scenario of a machining workshop, text data such as workshop production logs and equipment maintenance records, machine tool operation monitoring video data, and data collected by machine tool vibration and temperature sensors are input into the corresponding trained keyword extraction model. The model extracts keyword entities such as "lathe", "risk of tool wear", and "turn off the power before replacing the tool" from the text data; identifies the illegal operation of workers not wearing safety glasses from the video data and extracts keyword entities such as "risk of not wearing safety glasses"; detects abnormal vibration of the machine tool from the sensor data and extracts keyword entities such as "lathe" and "risk of abnormal vibration".

[0029] Meanwhile, according to the extracted keyword entities, they are classified into three categories: industrial equipment, risks, and operation steps. The keyword entities are classified accordingly and placed in the corresponding levels to obtain multi-level entities. In this embodiment, it is divided into a physical layer, a logical layer, and an operation layer. Among them, the physical layer is industrial equipment, including various machines, devices, etc. involved in the production process; the logical layer is risk keywords, including various potential dangerous situations related to equipment operation, personnel operation, etc.; the operation layer is operation steps, which are specific operation behaviors for equipment operation and risk prevention. For example, "injection molding machine", "stamping machine", etc. are classified into the physical layer; "risk of mold damage", "risk of high-temperature scald", etc. are classified into the logical layer; "check the mold temperature before injection molding", "check the mold before starting the stamping machine", etc. are classified into the operation layer. By stratifying the keyword entities, different types of information can be quickly located. When conducting risk analysis or formulating operation specifications, analyzing entities at different levels can quickly find relevant equipment, risk, and operation step information, improving information retrieval and analysis efficiency.

[0030] Furthermore, according to the different-level entities after division, by analyzing the association relationships between the level entities, the entities with association relationships are connected to construct an entity map. For example, there is a potential danger relationship between the physical layer entities and the logical layer entities, and there is a prevention and response relationship between the logical layer entities and the operation layer entities. By analyzing the association relationships between the entities at different levels, connections are established between the corresponding entities to construct the corresponding entity map. By constructing the entity map, the overall operation status during the industrial production process can be controlled, and the corresponding prevention measures can be quickly screened according to the dangerous situations, thereby improving the scientific nature of production management and the accuracy of decision-making, effectively reducing production risks, and enhancing production efficiency.

[0031] Furthermore, constructing the entity map by analyzing the association relationships between the entities at each layer, where the multi-level entities include a physical layer, a logical layer, and an operation layer, includes: S301. Obtain risk associations by analyzing the risk triggering probability between the operating status of the physical layer entities and the logical layer entities; S302. Obtain response associations by analyzing the risk response relationships between the logical layer entities and the operation layer entities; S303. Establish associations between the corresponding physical layer entities and the logical layer entities according to the risk associations, and establish associations between the corresponding logical layer entities and the operation layer entities according to the response associations to obtain the entity map.

[0032] In this embodiment, the association relationship between the physical layer entity and the logical layer entity is analyzed. By analyzing the risk triggering probability between the operating state of the physical layer entity and the logical layer entity, the risk association is obtained. The pre-trained random forest model is used to calculate the corresponding risk triggering probability. A large amount of historical production data is used, including records of various risks occurring in the equipment under different operating states, to train the random forest model and obtain the pre-trained random forest model. According to the input operating state data of the physical layer entity, the model calculates the probability values of triggering various risks in the logical layer through the learned mapping relationship between the physical layer entity and the logical layer entity. An association is established between entities with probability values greater than the preset threshold (in this embodiment, a probability value greater than 0.5 is regarded as a high association). By quantifying the relationship between the operating state of the physical layer entity and the logical layer risk, the risks that the equipment will trigger can be predicted in advance and prevented.

[0033] Furthermore, the association relationship between the logical layer entity and the operation layer entity is analyzed. By analyzing the response relationship between different risks in the logical layer entity and the operation layer entity, the corresponding response association is determined. According to professional knowledge in the industrial production field, industry standard documents, internal safety production regulations of the enterprise, and historical accident handling experience and other materials, the response measures and operation steps for different risks are extracted to form a risk response knowledge base. Each risk entity in the logical layer is compared with the content in the risk response knowledge base to find all effective response measures and operation steps corresponding to each risk, that is, the operation layer entity. The corresponding relationship between the logical layer entity and the operation layer entity is marked to determine which operation steps need to be taken to respond to each risk specifically. For example, for the "risk of high-temperature scald", operation layer entities such as "wear heat-insulating gloves" and "set high-temperature warning signs" are marked as response measures. An association is established between the corresponding entities. By clarifying the relationship between the logical layer risk and the operation layer response measures, the corresponding response measures can be quickly searched when the risk occurs, avoiding the expansion or improper handling of the risk due to unclear response measures.

[0034] As Figure 2 shown, an association is established between the corresponding physical layer entity and the logical layer entity according to the analyzed risk association, and an association is established between the corresponding logical layer entity and the operation layer entity according to the analyzed response association to obtain an entity map. By constructing the entity map, the risks faced in the industrial production process and the corresponding prevention measures can be comprehensively grasped, the production plan and resource allocation can be reasonably arranged, and the accuracy and safety of operations can be improved, thereby effectively enhancing the production control in the industrial production process.

[0035] Furthermore, according to the multi-modal data, through a preset hazard identification model, the hazard elements in the production process are identified, including: S401. Extract the features of the corresponding modal data from the multimodal data respectively to construct a joint feature vector. S402. Identify the dangerous conditions during the production process through a preset danger identification model based on the joint feature vector to obtain danger factors.

[0036] In this embodiment, first, according to the multimodal data in the production process, extract the features of the corresponding modal data from the preprocessed multimodal data respectively, and combine these features into a joint feature vector. For example, for text data, this embodiment uses the bag-of-words model, trains the bag-of-words model with a large amount of preprocessed text data to obtain a pre-trained bag-of-words model, and extracts the corresponding text features through the pre-trained bag-of-words model. For image data, this embodiment uses a convolutional neural network model, trains the convolutional neural network model with a large amount of image data to obtain a pre-trained convolutional neural network model, and extracts the corresponding image features through the pre-trained convolutional neural network model. Combine the extracted features of each modality in sequence to obtain a joint feature vector. By constructing a joint feature vector, the complementary nature of multimodal data can be fully utilized to improve the accuracy and comprehensiveness of danger factor identification. Data of different modalities can reflect the situation during the production process from different perspectives. For example, text data can provide information such as operation instructions and fault records, and image data can directly display the equipment status and personnel operation behaviors. After integrating the features of multimodal data, richer and more comprehensive features can be obtained, so as to more accurately identify potential danger factors and reduce the situations of missed detection and false detection.

[0037] Furthermore, according to the fused joint feature vector, identify the dangerous conditions during the production process through a preset danger identification model to obtain danger factors. The danger identification model can be a support vector machine, a random forest, a gradient boosting tree and other models. The danger identification model in this embodiment is specifically a random forest model. Train the random forest model with the joint feature vectors extracted from a large amount of historical multimodal data to obtain a pre-trained random forest model. Input the fused joint feature vector into the pre-trained random forest model, and the model outputs the prediction result of the dangerous condition. According to the model output, determine the danger factors during the production process, such as equipment failures, personnel's non-compliant operations, environmental anomalies, etc. By automatically identifying the dangerous conditions in the industrial production process, potential danger factors can be discovered in a timely manner. Using a danger identification model for identification can greatly improve the identification efficiency and accuracy.

[0038] Furthermore, the screening of the target entity chain matching the danger factor in the entity graph through a preset reordering model includes: S501. Construct an association matrix by analyzing the association relationship between the danger factor and each entity in the entity graph. S502. According to the association matrix, entity screening is performed in each layer of the entity graph through a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain.

[0039] In this embodiment, the association relationship between the hazard factor and each entity in the entity graph is analyzed, and an association matrix is constructed according to the corresponding association degree, where the association degree represents the tightness of the association between the hazard factor and each entity. Specifically, according to the hazard factor, each entity in the entity graph is traversed, and the association relationship between each entity and the hazard factor is analyzed in turn. Taking the entities in the entity graph as rows and the hazard factors as columns, the corresponding positions are filled according to the analyzed association degree to obtain the association matrix. By constructing the association matrix, the association situation between the hazard factor and each entity can be intuitively displayed. Compared with the traditional method of directly matching the corresponding countermeasures according to the hazard factor, this solution combines the association relationship between the hazard factor and each entity to screen the countermeasures, and can quickly screen out the entities closely related to the hazard factor, improving the accuracy and efficiency of the analysis.

[0040] Specifically, entities are screened according to the constructed association matrix. By analyzing the association degree between the hazard factor and each entity in the association matrix, entities with a higher association degree with the hazard factor are screened out in each layer of the entity graph. A preset reordering model is used to screen out the target entity chain that can reflect the causes, related risks, and countermeasures of the hazard factor. By screening out the target entity chain through the reordering model, entities closely related to the hazard factor can be quickly located from the entity graph, and targeted solutions can be formulated, thereby improving the hazard response speed and reducing the accident impact and losses. In this embodiment, the association matrix and the reordering model are combined to screen the target entity chain, and the entity chain closely related to the hazard factor can be accurately found from the entity graph, quickly locating the source of the hazard and saving time and resources.

[0041] Further, the construction of the association matrix by analyzing the association relationship between the hazard factor and each entity in the entity graph includes: S601. Extract corresponding features from the hazard factor to construct a first feature vector; S602. Map each entity to a second feature vector according to the state of each entity in the entity graph; S603. Calculate the similarity between the first feature vector and the second feature vector to obtain the association degree between the hazard factor and each entity; S604. Construct an initial association matrix according to the association degree; S605. Analyze the changes in the industrial production process environment of the hazard factor and the entity respectively within a preset time period, and update the initial association matrix to obtain the association matrix.

[0042] In this embodiment, first, corresponding features are extracted from the hazard elements. The hazard elements contain key information related to potential risks. Different types of hazard elements have different characteristic manifestations. For example, the hazard elements of equipment failure are reflected in the abnormal changes of equipment operation parameters, and the hazard elements of personnel's illegal operations are related to the behavior actions of personnel. These features are extracted and the first feature vector is constructed. Determine which category the current hazard element belongs to, such as types like equipment failure, personnel's illegal operations, environmental anomalies, etc.; according to the type of hazard element, feature extraction is performed on different types of hazard elements respectively. For the image information of the equipment in the hazard elements of equipment failure, the visual features of the image can be extracted using a convolutional neural network; for the hazard elements of personnel's illegal operations, the video data is analyzed, and the action features of personnel, such as limb positions, action amplitudes, etc., are extracted using a human pose recognition algorithm; the extracted features are integrated in sequence to obtain the first feature vector; for example, for a hazard element of equipment failure composed of equipment temperature, vibration data, and image features, the temperature mean, vibration variance, and image feature vector are sorted to obtain the first feature vector; by constructing the first feature vector, data support is provided for judging the association relationship between the hazard element and the entity.

[0043] Meanwhile, according to the status of each entity in the entity graph, each entity is mapped to a second feature vector. Entities in the entity graph have different attribute and status information. These status information of the entities are mapped into the same data form as the first feature vector to obtain the second feature vector. For each entity in the entity graph, determine the corresponding status information of the entity. For industrial equipment entities at the physical layer, the status information includes the operating parameters of the equipment, the service life of the equipment, maintenance records, etc.; for risk entities at the logical layer, the status information includes the historical frequency of risk occurrence, the severity level of the risk, etc.; for operation step entities at the operation layer, the status information includes the execution frequency of the step, the execution success rate, etc.; according to the type of entity status information, it is mapped into a vector form. For numerical status information, the numerical value is directly used as an element of the vector; for non-numerical information, such as the text description in the equipment maintenance record, the text vectorization method can be used to convert it into a numerical vector; for classification information, such as the severity level of the risk (high, medium, low), one-hot encoding can be used to convert it into a vector; the vectors after mapping the various status information of each entity are integrated to obtain the second feature vector corresponding to the entity. For example, for an industrial equipment entity, its operating parameter vector, service life numerical value, maintenance record text vector, etc. are connected in sequence to obtain the second feature vector of the equipment entity. By mapping the entity to the second feature vector, various status information of the entity can be comprehensively reflected. When calculating the correlation degree with hazard elements, the internal connection between the entity and the hazard elements can be more accurately reflected, avoiding analysis errors caused by inconsistent information representation, and improving the accuracy and reliability of the construction of the correlation matrix.

[0044] Specifically, calculate the similarity between the first feature vector and the second feature vector to obtain the correlation degree between the hazard element and each entity. The similarity calculation methods include Euclidean distance, cosine similarity, Manhattan distance, etc. In this embodiment, the cosine similarity between the first feature vector and the second feature vector is calculated to obtain the correlation degree between the hazard element and each entity. Through similarity calculation, the tightness of the correlation between the hazard element and the entity can be measured. The higher the similarity, the greater the correlation degree between the hazard element and the entity; according to the calculated correlation degree, construct an initial correlation matrix. The rows of the matrix represent the various entities in the entity graph, the columns represent the hazard elements, and the element values in the matrix are the correlation degrees between the corresponding entities and the hazard elements.

[0045] Specifically, within a preset time period, the initial association matrix is updated according to the risk factors, the status of entities, and the changes in the industrial production process environment. Considering that the risk factors and the status of entities in the industrial production process change over time, for example, equipment wear, production process adjustment, personnel changes, etc., which will affect the association relationship between risk factors and entities. According to the actual situation and requirements of industrial production, a time period is set, such as every hour, every day, etc. Within the time period, the information related to risk factors and the status information of entities in the entity map are monitored in real time; according to the monitored environmental changes, the first feature vector of the risk factors and the second feature vector of the entities are updated, and the corresponding association degrees are updated. The initial association matrix is updated using the updated association degrees to obtain the latest association matrix; by regularly updating the association matrix, the association matrix can adapt to the dynamic changes in the industrial production process, accurately reflect the real-time association relationship between risk factors and entities, avoid misjudgment of the association relationship caused by changes in the production environment, improve the enterprise's ability to respond to risks, and ensure the safe and stable operation of the production process.

[0046] Further, according to the association matrix, entity screening is performed in each layer of the entity map through a preset reordering model, and the screened entities in each layer are connected to obtain a target entity chain, including: S701. Respectively extract the association degrees corresponding to the entities in each layer of the entity map from the association matrix to obtain an association sub-matrix for each layer of entities; S702. According to the association sub-matrix, screen out the entities with association degrees greater than a preset association degree threshold as the candidate entity set for each layer; S703. Combining the connection relationships between the inter-layer entities in the entity map, screen out multiple connected entity chains from the candidate entity set to obtain a candidate entity chain set; S704. Through a preset reordering model, analyze the association degree of each entity chain in the candidate entity chain set, calculate a reordering score, and screen out the entity chain with the highest reordering score as the target entity chain.

[0047] In this embodiment, the correlation degrees corresponding to the entities at each layer are extracted from the correlation matrix to construct a correlation sub-matrix. The overall correlation matrix is disassembled by level, and the entities at each layer are screened separately. By dividing the correlation matrix into correlation sub-matrices, hierarchical screening is realized. There are differences in the correlation methods and influence degrees between the entities at different levels and the risk factors. Hierarchical processing can more accurately grasp the characteristics of the entities at each layer. According to the correlation sub-matrices, entities that are closely related to the risk factors and play an important role in the generation or response of risks are screened out from each correlation sub-matrix respectively. By selecting entities with a correlation degree greater than a preset correlation degree threshold as the candidate entity set for each layer, the correlation degree threshold is set based on the analysis of the actual production scenario and historical data. For example, through the statistical analysis of the correlation degrees between entities and risk factors in historical risk events, it is found that entities with a correlation degree higher than 0.6 play a key role in the occurrence and handling of risks, and the correlation degree threshold is set to 0.6. Each correlation sub-matrix is traversed, and the entities corresponding to the elements with a correlation degree greater than 0.6 are screened out to obtain the corresponding candidate entity set. By screening the candidate entities, entities that have little relationship with the risk factors can be quickly excluded, the screening speed can be accelerated, and the accuracy of judging the entities related to the risk factors can be improved.

[0048] Specifically, in the selected candidate entity set, according to the connection relationships between the candidate entities at different levels, multiple entity chains are obtained, and a candidate entity chain set is constructed. Starting from the candidate entities at the physical layer, the candidate entities at the logical layer connected to them are searched for, and then the corresponding candidate entities at the operation layer are further found to form a complete entity chain. All the qualified entity chains are summarized to obtain the candidate entity chain set. By screening out the candidate entity chain set, the causal relationships and response paths related to the risk factors are displayed in the form of chains.

[0049] Such as Figure 3As shown in the figure, the entity chain with the closest relationship to the risk factor is selected from the candidate entity chain set as the target entity chain. The association degree between each candidate entity chain and the risk factor is calculated respectively through a preset re-ranking model to obtain the corresponding score. The higher the score, the closer the relationship between the entity chain and the risk factor, and the more critical it is to the generation and handling of risks. The entity chain with the highest score is selected as the target entity chain. The re-ranking model can be a ranking algorithm based on gradient boosting, a neural network ranking model, etc. In this embodiment, the re-ranking model is specifically a neural network ranking model. A large amount of historical risk event data, including risk factors, corresponding entity graphs, generated candidate entity chain sets, and target entity chains identified in the actual handling process, are used to train the neural network ranking model to obtain a pre-trained re-ranking model. Each entity chain in the candidate entity chain set is input into the pre-trained re-ranking model. The model calculates the re-ranking score of each entity chain by combining factors such as the sum of the association degrees of each entity in the entity chain, the length of the chain, and the tightness of the inter-layer connection. The entity chain with the highest re-ranking score in the candidate entity chain set is selected as the target entity chain. By screening the target entity chain through the re-ranking model, the entity chain with the closest and most critical relationship to the risk factor can be accurately found. The target entity chain reflects the root cause of the risk, the involved risks, and effective countermeasures, which can improve the risk handling efficiency and reduce accident losses.

[0050] Further, matching the corresponding risk response plan in a preset safety operation knowledge base according to the target entity chain to perform safety control on the industrial field includes: S801. According to the target entity chain, perform matching in the preset safety operation knowledge base in hierarchical order to obtain the corresponding countermeasures; S802. Combine the association degrees between the entities in the target entity chain to dynamically combine the countermeasures to obtain the corresponding risk response plan for safety control of the industrial field.

[0051] In this embodiment, according to the selected target entity chain, it is matched in a preset safety operation knowledge base according to the hierarchical order of the target entity chain. The preset safety operation knowledge base stores countermeasures for various industrial production risks. According to the hierarchical division of the entity graph, starting from the physical layer, relevant countermeasures related to physical layer entities are searched in the safety operation knowledge base; then, countermeasures corresponding to logical layer entities are searched; finally, countermeasures for operation layer entities are searched. For example, for equipment entities in the physical layer, relevant measures under keywords such as "equipment maintenance" and "equipment inspection" are searched in the knowledge base; for risk entities in the logical layer, measures under keywords such as "risk prevention" and "risk warning" are searched. According to the hierarchical matching method, the matching is carried out in the safety operation knowledge base in turn. Using the entities in the target entity chain as keywords, matching or semantically similar countermeasures are searched under the corresponding classifications in the knowledge base. Matching countermeasures in the knowledge base according to the hierarchical order can avoid blind search and improve the matching efficiency. At the same time, the hierarchical matching method can ensure the corresponding relationship between the countermeasures and entities at each level in the target entity chain, making the obtained countermeasures more targeted and able to comprehensively cover the links involved in dangerous elements from multiple levels such as equipment operation, risk prevention, and operation execution, so as to formulate an effective danger response plan.

[0052] Specifically, the matched countermeasures are dynamically combined. By analyzing the degree of importance and mutual influence relationship of entities in the process of danger generation and development through the association degree between entities in the target entity chain, and dynamically combining these measures according to the entity association degree, the countermeasures for key links can be highlighted, and the measures for secondary links can be weakened, making the danger response plan more in line with the actual danger situation and enhancing the pertinence and effectiveness of the plan.

[0053] Further, by combining the association degree between entities in the target entity chain, dynamically combining the countermeasures to obtain a corresponding danger response plan for safety control of the industrial field, including: S901. Extract the total association degree between the corresponding entity and the connected entity from the association matrix according to the target entity chain; S902. Assign corresponding priority weights to each entity in the target entity chain according to the total association degree; S903. Dynamically combine the corresponding countermeasures according to the priority weights to obtain a corresponding danger response plan for safety control of the industrial field.

[0054] In this embodiment, according to the target entity chain, the total association degree between each entity in the target entity chain and the connected entity is extracted from the association matrix to quantify the importance and influence of each entity in the chain. The higher the total association degree, the more critical the role of the entity in the generation, development, or response process of the danger. For each entity in the target entity chain, find the entity directly connected to it in the entity graph, and extract the corresponding association degree value from the association matrix. Sum up the association degree values of each entity and the connected entity to obtain the total association degree of the entity. By extracting the total association degree, the importance of each entity in the target entity chain can be reflected, which helps to accurately grasp the key links of the danger and improve the pertinence and effectiveness of the danger response.

[0055] Specifically, according to the calculated total association degree, assign corresponding priority weights to each entity in the target entity chain. The higher the priority weight, the more important the corresponding response measure is in the danger response plan and needs to be executed first or focused on. By assigning priority weights, when formulating a danger response plan, the execution order and resource investment of the response measures can be determined to ensure that key issues are resolved first. In this embodiment, the normalization method is used to convert the total association degree into a priority weight value between 0 and 1, and calculate the priority weight of each entity in the target entity chain. By assigning priority weights, the importance order of each entity in the target entity chain can be determined, making the formulation of the danger response plan more logical and sequential, giving priority to dealing with key issues, and improving the resource utilization efficiency.

[0056] Specifically, after calculating the priority weights of each entity, the response measures are integrated according to the magnitude of the weight values to obtain a complete, orderly and targeted risk response plan. The response measures with high weights are executed first to ensure that the risks are effectively controlled. The response measures corresponding to each entity are sorted according to the priority weights of the corresponding entities, and the response measures corresponding to the entities with high weights are ranked in the front. According to the sorting results, the response measures are combined to obtain a preliminary risk response plan. For example, first list the response measures with high weights corresponding to the "risk of excessive pressure", and then list the measures corresponding to the "reactor" and "emergency pressure relief operation" in turn. Optimize the preliminarily combined risk response plan to determine the implementation time, implementation personnel, implementation conditions and other details of each response measure. At the same time, check the logic and feasibility of the plan to ensure that the measures cooperate with each other and work synergistically to obtain the final risk response plan. For example, it is stipulated that the "installation of a pressure alarm device" should be completed within 24 hours after the plan is started and is responsible for by the equipment maintenance department; the "regular inspection of the reactor seal" is carried out once a week and is executed by the production team, etc. By dynamically combining the response measures through the priority weights, the risk response plan closely focuses on the key links of the risk, improves the pertinence and effectiveness of the plan, reduces the possibility and harm degree of accidents. At the same time, the specific plan helps the operators to understand and execute, reduces the communication cost and operation errors, and improves the overall safety control ability and emergency response level of the enterprise.

[0057] Embodiment 2

[0058] In this embodiment, as Figure 4 , a safety control system for the industrial field combining a multi-modal model is provided for implementing the described method for the safety control of the industrial field combining a multi-modal model, including: A data acquisition module for acquiring multi-modal data in the production process of the industrial field; An entity graph construction module for respectively extracting industrial equipment, risks and operation steps from the multi-modal data as keyword entities, and constructing an entity graph by analyzing the association relationships between the keyword entities; A risk factor identification module for identifying risk factors in the production process according to the multi-modal data through a preset risk identification model; An entity chain screening module for screening out target entity chains matching the risk factors in the entity graph through a preset re-ranking model; A response plan formulation module for matching corresponding risk response plans in a preset safety operation knowledge base according to the target entity chains to perform safety control on the industrial field.

[0059] In this embodiment, the data acquisition module comprehensively and real-time collects multi-modal data in the production process of the industrial field, obtaining various types of data including text, images, video data, etc. For example, it obtains text data such as equipment operation logs and operation records from the production management system, and collects image and video data of equipment operation status and personnel operation behaviors through workshop cameras. The collected data undergoes preliminary cleaning and preprocessing to remove noise and invalid data, providing data support for safety analysis in the industrial field; the entity graph construction module, based on the multi-modal data provided by the data acquisition module, mines key information in the industrial production process, constructs a corresponding network, extracts industrial equipment, risks, and operation steps as keyword entities from the multi-modal data, divides the keyword entities into different levels according to industrial equipment, risks, and operation steps to form a multi-layer entity structure, and constructs an entity graph by analyzing the association relationships between entities in each layer. The entity graph shows the internal connections between various elements in the industrial production process, providing a knowledge framework for subsequent hazard analysis and response decision-making; the hazard element identification module, according to the multi-modal data, through a preset hazard identification model, identifies the hazardous conditions in the production process, extracts corresponding features from the multi-modal data respectively to construct feature vectors, inputs the feature vectors into the hazard identification model. The model analyzes and judges the current production process by learning the mapping relationship between features and hazardous conditions in historical data, and identifies hazard elements, including hazardous conditions such as equipment failures, personnel's illegal operations, and environmental abnormalities. By identifying hazard elements, potential safety hazards can be discovered in advance, and preventive measures can be taken to reduce the probability of accidents.

[0060] Specifically, the entity chain screening module, based on the entity graph, screens out target entity chains related to hazard elements. By analyzing the association relationships between hazard elements and each entity in the entity graph, an association matrix is constructed to obtain the association degree between hazard elements and each entity. Combining the connection relationships between inter-layer entities in the entity graph, the entity chains are screened to obtain the target entity chain with the highest matching degree with the hazard elements. By screening out the target entity chain, it can provide a direction for formulating a hazard response plan; the response plan formulation module, according to the screened target entity chain, matches and generates a corresponding hazard response plan in a preset safety operation knowledge base. According to the hierarchical order of the target entity chain, it is matched in the safety operation knowledge base to obtain response measures corresponding to each entity in the target entity chain. Combining the association degrees between entities in the target entity chain, the matched response measures are dynamically combined to form a complete, orderly, and targeted hazard response plan. Responding quickly to the hazardous conditions according to the hazard response plan can reduce accident losses and ensure the safe and stable operation of industrial production.

[0061] Embodiment 3

[0062] In this embodiment, a specific implementation manner of an industrial field safety control method combined with a multi-modal model is provided. As Figure 5 shown, first, a vectorization tool small model is used to structurally process safety management documents such as industry standards and enterprise specifications. This tool uses a pre-trained deep learning model to convert text content into high-dimensional vectors and constructs a "safety management document vector knowledge base", and the knowledge base supports dynamic updates to ensure synchronization with the latest regulations and standards.

[0063] When the user inputs a risk-related query through natural language questions, the system realizes semantic understanding and context association based on the AI multi-modal large model. The large model combines the "safety management document vector knowledge base" for retrieval, filters key information through the attention mechanism, and generates accurate answers that conform to industry terms. The AI multi-modal large model supports multi-modal inputs such as text, images, and structured data, which can enhance the adaptability to complex scenarios.

[0064] The system utilizes the zero-shot or few-shot learning ability of the large model to analyze equipment parameters, environmental data, historical accident records, etc., and automatically identify major hazard sources; by comparing with the risk feature library in the knowledge base (such as chemical flash points and pressure vessel thresholds), it outputs a list of hazard sources and their potential hazard chains.

[0065] Based on the identification results, the system calls a pre-set risk matrix algorithm (such as the LEC method or the MES method), combines the quantitative evaluation of uncertain factors by the large model (such as the probability of human operation errors), generates a dynamic risk level, and introduces an adversarial training mechanism in the assessment process to ensure the correctness of the results in extreme scenarios.

[0066] Through the generative AI ability of the AI multi-modal large model, it outputs hierarchical control measures, including technical improvements, emergency plans, training requirements, etc. The measure generation adopts a reinforcement learning framework, continuously absorbs the feedback of the actual control effect for iterative optimization, and forms a closed-loop management.

[0067] As Figure 6 shown, by using the vectorization tool small model, feature extraction and vector encoding are performed on structured texts such as "established safety inspection plans" and "hidden danger investigation standards" to populate the "safety management document vector knowledge base". This embodiment uses a lightweight model based on Transformer to achieve high-compression semantic representation, ensuring high accuracy of the content in the knowledge base.

[0068] Through the multi-modal data such as inspection images, videos, and audios real-time transmitted by on-site terminals, the system conducts cross-modal alignment and feature fusion through an AI multi-modal large model, uses spatio-temporal attention mechanisms to analyze the operating states of equipment and the working states of personnel in videos, and simultaneously combines voiceprint recognition technology to conduct audio anomaly analysis, such as mechanical abnormal noises, pipeline leakage whistles, equipment roars, etc., to achieve joint modeling of multi-dimensional data.

[0069] The AI multi-modal large model compares the retrieval results of the knowledge base with real-time data analysis, outputs the hidden danger location and risk type judgment. The confidence probability is calculated through the Monte Carlo Dropout technology, the result self-evaluation is completed, and a quantitative evaluation report is generated according to relevant national standards. When the confidence is lower than the preset threshold, the manual review process is automatically triggered. For the diagnosis results, a hierarchical hidden danger warning plan is output, and the plan includes emergency disposal steps, resource scheduling suggestions, standardized operation guidelines, etc., and is pushed to the mobile terminal and the central control platform synchronously.

[0070] As Figure 7 shown, through the AI multi-modal large model, the pre-organized hidden danger rectification standards, historical case libraries, and benefit analysis reports are transformed into structured vectors through a quantization tool small model and filled into the "Safety Management Document Vector Knowledge Base"; the management method of this embodiment adopts the FAISS technology, that is, approximate nearest neighbor search, to achieve rapid matching of articles, specifications, treatment measures, cases, etc. For the determined hidden dangers such as multi-modal inputs of text descriptions, on-site images, sensor data, etc., the AI multi-modal large model conducts feature fusion analysis, extracts the equipment defect features in the image through the visual function, and simultaneously combines the keyword entities in the text description to retrieve historical cases and corresponding standard clauses with a similarity higher than the threshold in the "Safety Management Document Vector Knowledge Base".

[0071] After the hidden danger risk is triggered when exceeding the threshold, combined with the analysis ability of the AI multi-modal large model for the confirmed hidden dangers, rectification measures are generated: Level 1 measures: immediately execute measures, such as emergency disposals like "shutdown and pressure relief"; Level 2 measures: execute within 24 hours, such as technical means like "ultrasonic thickness measurement detection"; Level 3 measures: long-term improvement, such as engineering transformations like "anti-corrosion coating upgrade". At the same time, in the safety management method of the present invention, the rectification plan can be automatically associated with the material list through the analysis of the Agent assistant, and at the same time, operation guidance and preparation of a cost-benefit analysis report are carried out. Manual verification is carried out for the rectification measures provided by the multi-modal AI model, and manual acceptance is carried out after verification, and finally the hidden danger rectification is completed.

[0072] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only to illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. An industrial field safety control method combining a multi-modal model, characterized in that, Including: Obtaining multi-modal data in the production process of the industrial field; Respectively extracting industrial equipment, risks, and operation steps from the multi-modal data as keyword entities, and constructing an entity graph by analyzing the association relationships between the keyword entities; Identifying dangerous elements in the production process according to the multi-modal data through a preset danger identification model; Filtering out target entity chains matching the dangerous elements in the entity graph through a preset re-ranking model; Matching corresponding danger response plans in a preset safety operation knowledge base according to the target entity chain to perform safety control on the industrial field.

2. The industrial field safety control method combining a multi-modal model according to claim 1, characterized in that, The step of respectively extracting industrial equipment, risks, and operation steps from the multi-modal data as keyword entities, and constructing an entity graph by analyzing the association relationships between the keyword entities includes: Inputting the multi-modal data into a preset keyword extraction model respectively to extract industrial equipment, risks, and operation steps as keyword entities; Dividing the keyword entities into different levels according to industrial equipment, risks, and operation steps to obtain multi-layer entities; Constructing an entity graph by analyzing the association relationships between each layer of entities.

3. The industrial field safety control method combining a multi-modal model according to claim 2, wherein The step of constructing an entity graph by analyzing the association relationships between each layer of entities, where the multi-layer entities include a physical layer, a logical layer, and an operation layer, includes: Obtaining risk associations by analyzing the risk triggering probabilities between the operating states of physical layer entities and logical layer entities; Obtaining response associations by analyzing the risk response relationships between logical layer entities and operation layer entities; Establishing associations between corresponding physical layer entities and logical layer entities according to the risk associations, and establishing associations between corresponding logical layer entities and operation layer entities according to the response associations to obtain an entity graph.

4. An industrial field safety control method combining a multi-modal model according to claim 1, characterized in that, The step of identifying dangerous elements in the production process according to the multi-modal data through a preset danger identification model includes: Respectively extracting features of corresponding modal data from the multi-modal data to construct a joint feature vector; Identifying the dangerous conditions in the production process through a preset danger identification model according to the joint feature vector to obtain dangerous elements.

5. The industrial field safety control method combining a multi-modal model according to claim 1, characterized in that, The step of filtering out target entity chains matching the dangerous elements in the entity graph through a preset re-ranking model includes: Constructing an association matrix by analyzing the association relationships between the dangerous elements and each entity in the entity graph; Performing entity screening in each layer of the entity graph according to the association matrix through a preset re-ranking model, and connecting the screened entities in each layer to obtain a target entity chain.

6. The industrial field safety control method combining a multi-modal model according to claim 5, characterized in that, The step of constructing an association matrix by analyzing the association relationships between the dangerous elements and each entity in the entity graph includes: Extracting corresponding features from the dangerous elements to construct a first feature vector; Mapping each entity to a second feature vector according to the state of each entity in the entity graph; Calculating the similarity between the first feature vector and the second feature vector to obtain the association degree between the dangerous elements and each entity; Constructing an initial association matrix according to the association degree; Within a preset time period, analyze the changes in the industrial production process environment of the risk factors and entities respectively, and update the initial association matrix to obtain an association matrix.

7. The industrial field safety control method combining a multi-modal model according to claim 5, wherein, According to the association matrix, perform entity screening in each layer of the entity graph through a preset reordering model, and connect the screened entities in each layer to obtain a target entity chain, including: Extract the association degrees corresponding to the entities in each layer of the entity graph from the association matrix respectively to obtain an association sub-matrix for each layer of entities; According to the association sub-matrix, screen out the entities with association degrees greater than the preset association degree threshold as the candidate entity set for each layer; Combined with the connection relationships between the inter-layer entities in the entity graph, screen out multiple connected entity chains from the candidate entity set to obtain a candidate entity chain set; Through a preset reordering model, analyze the association degree of each entity chain in the candidate entity chain set, calculate the reordering score, and screen out the entity chain with the highest reordering score as the target entity chain.

8. A safety control and management method for the industrial field combining a multi-modal model according to claim 1, characterized in that, According to the target entity chain, match the corresponding risk response plan in a preset safety operation knowledge base to conduct safety control over the industrial field, including: According to the target entity chain, perform matching in the preset safety operation knowledge base in hierarchical order to obtain corresponding countermeasures; Combined with the association degrees between the entities in the target entity chain, dynamically combine the countermeasures to obtain the corresponding risk response plan to conduct safety control over the industrial field.

9. A safety control and management method in the industrial field combining a multi-modal model according to claim 8, characterized in that, The step of dynamically combining the countermeasures according to the association degrees between the entities in the target entity chain to obtain the corresponding risk response plan to conduct safety control over the industrial field includes: According to the target entity chain, extract the total association degree between the corresponding entity and the connected entity from the association matrix; According to the total association degree, assign corresponding priority weights to each entity in the target entity chain; According to the priority weights, dynamically combine the corresponding countermeasures to obtain the corresponding risk response plan to conduct safety control over the industrial field.

10. An industrial field safety control system combined with a multi-modal model, characterized in that, An industrial field safety control method combining a multimodal model for implementing any one of claims 1 to 9 includes: A data acquisition module that acquires multimodal data in the industrial production process; An entity graph construction module that extracts industrial equipment, risks, and operation steps as keyword entities from the multimodal data respectively, and constructs an entity graph by analyzing the association relationships between the keyword entities; A risk factor identification module that identifies risk factors in the production process according to the multimodal data through a preset risk identification model; An entity chain screening module that screens out a target entity chain matching the risk factors in the entity graph through a preset reordering model; A countermeasure formulation module that matches the corresponding risk response plan in a preset safety operation knowledge base according to the target entity chain to conduct safety control over the industrial field.

Images