Industrial production monitoring method and system based on large model, terminal and storage medium

Abstract

The invention relates to an industrial production monitoring method and system based on a large model, a terminal and a storage medium, and relates to the technical field of industrial safety production monitoring, and the method comprises the steps: obtaining production multi-modal data of a preset production region; controlling a preset industrial safety monitoring model according to the production multi-modal data to generate an industrial safety production state through reasoning; judging whether the industrial safety production state meets the requirements of a preset normal production state or not; if yes, continuously obtaining the production multi-modal data of the production area for cyclic judgment; if not, analyzing the industrial safety production state to determine a production assessment risk; determining real-time alarm data according to the production assessment risk, the industrial safety production state and a preset risk alarm relationship; and performing alarm prompt according to the real-time alarm data. The method has the effect of improving the accuracy of industrial safety production monitoring.

Description

Technical Field

[0001] The present application relates to the field of industrial production safety monitoring technology, and in particular to industrial production monitoring methods, systems, terminals and storage media based on large models. Background Art

[0002] Industrial production safety refers to the process of industrial production, through a series of management and technical measures to ensure the safety of production activities and prevent accidents such as casualties, property losses and environmental pollution.

[0003] In related technologies, industrial production safety usually adopts video surveillance systems. By installing high-definition cameras in the production area, the working environment can be monitored uninterruptedly. Then, using intelligent video analysis technology, the system automatically identifies unsafe behavior patterns, such as not wearing safety equipment, illegally entering dangerous areas, etc., and immediately issues an alarm to notify relevant personnel.

[0004] Regarding the above-mentioned related technologies, video surveillance and video analysis technologies are used to monitor industrial production safety. The single modal data of visual analysis is difficult to capture complete scene information. For example, important safety features such as environmental noise and equipment vibration cannot be included in the analysis, making it impossible to efficiently identify potential safety hazards in complex scenes. This leads to low accuracy in industrial production safety monitoring and there is still room for improvement. Summary of the invention

[0005] In order to improve the accuracy of industrial production safety monitoring, the present application provides an industrial production monitoring method, system, terminal and storage medium based on a large model.

[0006] In the first aspect, the present application provides an industrial production monitoring method based on a large model, which adopts the following technical solutions: Industrial production monitoring methods based on large models include: Obtain production multimodal data of the preset production area; Generate industrial safety production status by inference based on the preset industrial safety monitoring model of production multimodal data control; Determine whether the industrial safety production status meets the requirements of the preset normal production status; If it meets the requirements, continue to obtain the multimodal production data of the production area for cyclic judgment; If not, the industrial safety production status is analyzed to determine the production assessment risk; Determine real-time alarm data based on production assessment risk, industrial safety production status and preset risk alarm relationship; Provide alarm prompts based on real-time alarm data.

[0007] By adopting the above technical solution, the production multimodal data of the production area is detected, and the industrial safety monitoring model is controlled to infer the industrial safety production status according to the production multimodal data, so as to fully present the production scene information with multi-modal data, thereby improving the accuracy of industrial safety production monitoring. When it is determined that the industrial safety production status does not meet the requirements of the normal production status, the real-time alarm data is determined according to the production assessment risk, the industrial safety production status and the risk alarm relationship, so as to make timely prompts when a safety accident occurs to reduce losses.

[0008] Optionally, the step of obtaining production multimodal data of a preset production area includes: Obtain the area usage serial number and equipment usage serial number of the production area; Determine the starting area device serial number according to the area use serial number and the preset area serial number device relationship; Determine the device serial number of the startup device according to the device usage serial number and the preset device serial number device relationship; According to the device serial number of the starting area and the device serial number of the starting equipment, the preset multimodal data acquisition device is controlled to collect the production data of the production area to generate production multimodal data.

[0009] By adopting the above technical solution, the starting area device number is determined according to the area usage number, and the starting equipment device number is determined according to the equipment usage number, so that the multimodal data acquisition device is controlled to collect and generate production multimodal data according to the starting area device number and the starting equipment device number, instead of starting all the devices. On the one hand, energy is saved, and on the other hand, the pressure on processing and storage of large amounts of collected data is reduced.

[0010] Optionally, the step of controlling a preset multimodal data collection device to collect production data of the production area according to the start-up area device serial number and the start-up equipment device serial number to generate production multimodal data includes: Controlling the multimodal data acquisition device to collect production data of the production area according to the device serial number of the start-up area and the device serial number of the start-up equipment to generate basic multimodal data; Performing data preprocessing on the basic multimodal data according to a preset data preprocessing method to generate processed multimodal data; Controlling a preset encoder to encode the processed multimodal data to generate encoded multimodal data; The encoded multimodal data is processed according to a preset cross-modal interactive fusion method to generate production multimodal data.

[0011] By adopting the above technical scheme, the multimodal data acquisition device is controlled to collect basic multimodal data, and then the basic multimodal data is preprocessed to obtain processed multimodal data, so as to ensure the uniformity of the data format, scale and timestamp, reduce the dimension of the data, and improve the calculation efficiency. Then, the processed multimodal data is encoded to generate encoded multimodal data, so as to ensure that the multimodal data can effectively interact and fuse within the same model. Finally, the encoded multimodal data is processed according to the cross-modal interactive fusion method to obtain production multimodal data, so as to establish the connection of modal quality inspection, enhance the understanding of complex scenes, and integrate the multimodal data into a unified representation to facilitate subsequent model reasoning.

[0012] Optionally, the steps of analyzing the industrial safety production status to determine the production assessment risk include: Analyze the industrial safety production status to determine the type of abnormal status; Determine whether the abnormal state type meets the requirements of the preset extreme abnormal type; If it meets the requirements, the production assessment risk is determined according to the preset comparison determination method; If not, analyze the industrial safety production status to determine the abnormal environmental status and equipment status; Determine environmental assessment risks based on the relationship between environmental abnormalities and preset environmental status risks; Determine the equipment assessment risk based on the equipment abnormal status and the preset equipment status risk relationship; Associate environmental assessment risks and equipment assessment risks to generate production assessment risks.

[0013] By adopting the above technical scheme, when the industrial production safety status meets the requirements of the extreme abnormal type, the production assessment risk is determined according to the comparison determination method. When it does not meet the requirements, the environmental assessment risk is determined according to the environmental abnormal state and the equipment assessment risk is determined according to the equipment abnormal state. After associating the two, the production assessment risk is obtained, thereby improving the convenience of determining the production assessment risk.

[0014] Optionally, the step of determining the production assessment risk according to a preset comparison determination method includes: Analyze the industrial safety production status to determine the first behavior status type and extreme abnormal status; Determine whether the first behavior state type meets the requirements of the preset leaving area behavior type; If it is in compliance, the production assessment risk is determined based on the extreme abnormal state and the preset extreme state risk relationship; If not met, the preset top production risk is defined as the production assessment risk.

[0015] By adopting the above technical solution, when it is determined that the first behavior state type meets the behavior type of leaving the area, it indicates that the person has left the area. Therefore, the production assessment risk is determined only based on the extreme abnormal state and the extreme state risk relationship. When it does not meet the requirements, it indicates that the person is still in the area. At this time, the top production risk is defined as the production assessment risk, thereby improving the accuracy of determining the production assessment risk.

[0016] Optionally, the step of determining the environmental assessment risk according to the abnormal environmental state and the preset environmental state risk relationship includes: Analyze the abnormal state of the environment to determine the type of the second behavior state; Determine whether the second behavior state type meets the requirements of the preset leaving area behavior type; If not, the environmental assessment risk is determined based on the preset behavioral risk determination method; If it meets the requirements, the preset unmanned risk is defined as the environmental assessment risk.

[0017] By adopting the above technical solution, when it is determined that the second behavior state type does not meet the requirements of the leaving area behavior type, it means that when the environment is abnormal, the personnel are still in the production area. Therefore, the environmental assessment risk is determined according to the behavior risk determination method. When it meets the requirements, it indicates that the personnel have left the production area where the environment is abnormal. Therefore, the definition of no-man risk is reduced to the environmental assessment risk, thereby improving the accuracy of determining the environmental assessment risk.

[0018] Optionally, the steps of determining environmental assessment risks according to a preset behavioral risk determination method include: Determining whether the second behavior state type meets the requirements of the preset non-dangerous behavior type; If not, the preset top production risk is defined as the environmental assessment risk; If it meets the requirements, the environmental assessment risk will be determined based on the relationship between the abnormal environmental state and the environmental state risk.

[0019] By adopting the above technical scheme, when the second behavior state type meets the requirements of the non-dangerous behavior type, it indicates that the personnel are wearing protective equipment neatly, and therefore the environmental assessment risk is determined based on the relationship between the abnormal environmental state and the environmental state risk. When it does not meet the requirements, it indicates that the personnel are not wearing protective equipment in the abnormal environment, and therefore the top production risk is defined as the environmental assessment risk, thereby improving the accuracy of determining the environmental assessment risk.

[0020] In the second aspect, the present application provides an industrial production monitoring system based on a large model, which adopts the following technical solutions: Industrial production monitoring system based on large model, including: An acquisition module, used to acquire production multimodal data; A memory for storing a program of the industrial production monitoring method based on a large model as described in any one of the above items; The program in the processor memory can be loaded and executed by the processor to implement the industrial production monitoring method based on the large model as described in any one of the above items.

[0021] By adopting the above technical solution, the processor loads and executes the program of the large model-based industrial production monitoring method stored in the memory, so that the acquisition module obtains a series of data related to the large model-based industrial production monitoring, thereby controlling the industrial safety monitoring model to infer the industrial safety production status according to the production multimodal data, and fully present the production scene information with multiple modal data, thereby improving the accuracy of industrial safety production monitoring. When it is determined that the industrial safety production status does not meet the requirements of the normal production status, the real-time alarm data is determined according to the production assessment risk, the industrial safety production status and the risk alarm relationship, so as to make timely prompts when a safety accident occurs to reduce losses.

[0022] In a third aspect, the present application provides a smart terminal, which adopts the following technical solution: An intelligent terminal comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes any one of the above-mentioned large-model-based industrial production monitoring methods.

[0023] By adopting the above technical solution and operating the intelligent terminal, the processor is made to load and execute the computer program of the industrial production monitoring method based on the large model stored in the memory, thereby controlling the industrial safety monitoring model to infer the industrial production safety status according to the production multimodal data, and fully present the production scene information with multiple modal data, thereby improving the accuracy of industrial production safety monitoring. When it is determined that the industrial production safety status does not meet the requirements of the normal production status, the real-time alarm data is determined according to the production assessment risk, the industrial production safety status and the risk alarm relationship, so as to make prompts in time when a safety accident occurs to reduce losses.

[0024] In a fourth aspect, the present application provides a computer storage medium capable of storing corresponding programs, which is convenient for improving the accuracy of industrial production safety monitoring, and adopts the following technical solutions: A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the above-mentioned industrial production monitoring method based on a large model.

[0025] By adopting the above technical solution, a computer program of an industrial production monitoring method based on a large model is stored in a computer-readable storage medium, so that the processor loads and executes the computer program in the storage medium, thereby controlling the industrial safety monitoring model to infer the industrial production safety status according to the production multimodal data, and fully present the production scene information with multiple modal data, thereby improving the accuracy of industrial production safety monitoring. When it is determined that the industrial production safety status does not meet the requirements of the normal production status, real-time alarm data is determined based on the production assessment risk, industrial production safety status and risk alarm relationship, so as to make timely prompts when a safety accident occurs to reduce losses.

[0026] In summary, the present application includes at least one of the following beneficial technical effects: By controlling the industrial safety monitoring model to infer the industrial safety production status based on the production multi-modal data, the production scene information is fully presented with multi-modal data, thereby improving the accuracy of industrial safety production monitoring. When it is determined that the industrial safety production status does not meet the requirements of the normal production status, the real-time alarm data is determined based on the production assessment risk, industrial safety production status and risk alarm relationship, so as to give prompts in time when a safety accident occurs to reduce losses; By determining the start-up area device serial number according to the area usage serial number, and determining the start-up equipment device serial number according to the equipment usage serial number, the multimodal data collection device is controlled to collect and generate production multimodal data according to the start-up area device serial number and the start-up equipment device serial number, instead of starting all the devices, which saves energy on the one hand and reduces the pressure on processing and storage of a large amount of collected data on the other hand; The basic multimodal data is collected by controlling the multimodal data acquisition device, and then the basic multimodal data is preprocessed to obtain the processed multimodal data, so as to ensure the uniformity of the data format, scale and timestamp, reduce the dimension of the data, and improve the calculation efficiency. The processed multimodal data is then encoded to generate encoded multimodal data, so as to ensure that the multimodal data can effectively interact and fuse within the same model. Finally, the encoded multimodal data is processed according to the cross-modal interactive fusion method to obtain the production multimodal data, so as to establish the connection of modal quality inspection, enhance the understanding of complex scenes, and integrate the multimodal data into a unified representation to facilitate subsequent model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the industrial production monitoring method based on a large model in an embodiment of the present application.

[0028] Figure 2 It is a flowchart of the steps of obtaining production multimodal data of a preset production area in an embodiment of the present application.

[0029] Figure 3It is a flowchart of the steps of controlling a preset multimodal data acquisition device to collect production data of a production area to generate production multimodal data according to a startup area device serial number and a startup equipment device serial number in an embodiment of the present application.

[0030] Figure 4 It is a flowchart of the steps of analyzing the industrial production safety status to determine the production assessment risk in an embodiment of the present application.

[0031] Figure 5 It is a flowchart of the steps of determining production assessment risks according to a preset comparison determination method in an embodiment of the present application.

[0032] Figure 6 It is a flowchart of the steps of determining environmental assessment risks according to the relationship between environmental abnormality and preset environmental status risks in an embodiment of the present application.

[0033] Figure 7 It is a flowchart of the steps of determining environmental assessment risks according to a preset behavioral risk determination method in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-Figure 7 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The embodiment of the present application discloses an industrial production monitoring method based on a large model, and specifically discloses a processing terminal and a multimodal data acquisition device. The processing terminal is compositely connected to the multimodal data acquisition device through data conductors and wireless communications to realize data interaction and control. After the processing terminal controls the multimodal data acquisition device to collect production multimodal data of the production area, the processing terminal controls the industrial safety monitoring model to infer the industrial production safety status according to the production multimodal data, and compares the industrial production safety status with the normal production status. If it is consistent with the normal production status, the production multimodal data continues to be collected for continuous monitoring. If it is inconsistent, it indicates that an abnormal state has occurred in the production area. Therefore, the industrial production safety status is analyzed to determine the production assessment risk, and then the real-time alarm data is determined according to the production assessment risk, the industrial production safety status and the risk alarm relationship, and prompts are given with the real-time alarm data, so as to respond to the safety accidents in a timely and accurate manner and minimize the losses.

[0036] Reference Figure 1 The present application embodiment discloses an industrial production monitoring method based on a large model, comprising the following steps: Step S100: Acquire production multimodal data of a preset production area.

[0037] The production area refers to the area where personnel control production equipment for industrial production. The specific area is determined by the operator based on the actual situation of the factory. Production multimodal data refers to the data of different modes related to the process of personnel controlling production equipment for production, including image, sound, temperature, vibration and other data obtained after preprocessing, encoding and fusion. For specific acquisition methods, refer to Figure 2 steps.

[0038] Step S101: Inferring and generating industrial safety production status based on the preset industrial safety monitoring model controlled by production multimodal data.

[0039] Among them, after the processing terminal receives the production multimodal data, the industrial safety monitoring model is controlled to perform inference with the production multimodal data as input, so as to conduct all-round perception and intelligent analysis of the industrial production environment to determine the industrial production safety status and provide data support for the subsequent determination of whether the industrial production is safe.

[0040] The industrial safety monitoring model refers to a large model for monitoring industrial production safety. In the embodiments of this application, a multimodal Transformer model is taken as an example. Combined with historical data of industrial production safety, such as video abnormal event records, equipment operation data, etc., the large model is fine-tuned and optimized to enable it to have high-precision safety hazard detection capabilities, and support incremental training with a small amount of new data to adapt to different industrial scenarios, online learning, and continuous optimization of the ability to identify unseen safety hazards.

[0041] The industrial safety production status refers to the semantic understanding of the production status of the production area, such as the environment and equipment are normal, or the environment is high temperature and high noise, and the equipment is operating normally, etc. It is determined by the industrial safety monitoring model after reasoning and analysis using production multimodal data as input.

[0042] Step S102: Determine whether the industrial safety production status meets the requirements of the preset normal production status.

[0043] Among them, the normal production status refers to the semantic understanding that the equipment and environment are normal, which is stored in the processing terminal by the operator, and the requirement of the normal production status refers to consistency with the normal production status.

[0044] By processing the terminal, it is determined whether the industrial production safety status is consistent with the normal production status, thereby determining whether the industrial production is in a safe production state.

[0045] Step S1021: If it meets the requirements, continue to obtain the multimodal production data of the production area for cyclic judgment.

[0046] Among them, if the processing terminal determines that the industrial production safety status is consistent with the normal production status, it means that the industrial production is in a safe production state. Therefore, the production multimodal data of the production area continues to be detected, thereby continuously monitoring the production and life in the production area.

[0047] Step S1022: If not, analyze the industrial production safety status to determine the production assessment risk.

[0048] Among them, if the processing terminal determines that the industrial production safety status is inconsistent with the normal production status, it means that the industrial production is in a dangerous state. Therefore, after analyzing the industrial production safety status, the production assessment risk is determined to provide data support for subsequent alarm prompts.

[0049] Production assessment risk refers to the risk level of industrial production, which is determined by the processing terminal based on the industrial safety production status analysis. The specific analysis method is referred to Figure 4 steps.

[0050] Step S103: Determine real-time alarm data according to production assessment risk, industrial production safety status and preset risk alarm relationship.

[0051] Among them, the risk alarm relationship refers to the correspondence between the risk level and the alarm type. In the embodiment of the present application, the higher the risk level, the more urgent the light and sound corresponding to the alarm type. The operator forms a mapping table by matching the risk level and the alarm type one by one.

[0052] Real-time alarm data refers to data that alerts the current dangerous production status. The processing terminal looks up the alarm type in the mapping table corresponding to the risk alarm relationship based on the production assessment risk, and then associates the alarm type with the semantic understanding corresponding to the industrial safety production status to generate real-time alarm data.

[0053] Step S104: Produce an alarm prompt based on the real-time alarm data.

[0054] Among them, after the processing terminal obtains real-time alarm data, the processing terminal controls the alarm light to flash with the light corresponding to the real-time alarm data, and prompts with an alarm sound. At the same time, the semantic understanding corresponding to the industrial safety production status is sent to the terminal held by the personnel for the personnel to view and solve safety problems.

[0055] Reference Figure 2 , the steps of obtaining the production multimodal data of the preset production area include: Step S200: Obtain the area usage serial number and equipment usage serial number of the production area.

[0056] Among them, the area usage number refers to the serial number of the personnel using the production area, and the equipment usage number refers to the serial number of the personnel using the production equipment. The operator numbers the production area and production equipment, and enters the serial number of the area and equipment to be used through the handheld terminal.

[0057] Step S201: determining the startup area device serial number according to the area usage serial number and the preset area serial number device relationship.

[0058] The area number-device relationship refers to the correspondence between the area number and the area device. The operator forms a mapping table by mapping the area number and the area device one by one according to the actual situation of the production area.

[0059] The startup regional device serial number refers to the serial number of the regional device that needs to be started, such as a camera, a microphone, a thermometer and other devices, which is found by the processing terminal in the mapping table corresponding to the regional serial number device relationship according to the regional usage serial number.

[0060] Step S202: Determine the startup device serial number according to the device usage serial number and the preset device serial number device relationship.

[0061] The equipment number device relationship refers to the corresponding relationship between the equipment number and the equipment device. The operator forms a mapping table by matching the equipment device number on each production equipment with the equipment number one by one.

[0062] The startup equipment device serial number refers to the serial number of the equipment device that needs to be started, such as a vibration sensor used to detect vibration of production equipment, etc., which is found by the processing terminal in the mapping table corresponding to the equipment serial number device relationship according to the equipment usage serial number.

[0063] Step S203: According to the device serial number of the starting area and the device serial number of the starting equipment, a preset multimodal data acquisition device is controlled to acquire production data of the production area to generate production multimodal data.

[0064] Among them, after the processing terminal obtains the device serial number of the starting area and the device serial number of the starting equipment, the processing terminal controls the multimodal data acquisition device with the corresponding serial number to collect the production data of the production area to obtain the production multimodal data. The specific method is referred to Figure 3 steps.

[0065] A multimodal data acquisition device refers to a device used to collect data of different modes in a production area, including area devices, such as cameras, thermometers, and microphones, etc., and equipment devices, such as vibration sensors.

[0066] Reference Figure 3The step of controlling a preset multimodal data acquisition device to collect production data of a production area according to the device serial number of the start-up area and the device serial number of the start-up equipment to generate production multimodal data includes: Step S300: Control the multimodal data acquisition device to collect production data of the production area according to the startup area device serial number and the startup equipment device serial number to generate basic multimodal data.

[0067] Among them, the processing terminal controls the regional devices in the multimodal data acquisition device according to the starting regional device serial number, such as cameras, microphones, thermometers and other devices to collect production data of the production area, and controls the equipment devices in the modal data acquisition device according to the starting equipment device serial number, such as vibration sensors to collect production data of the production area, thereby obtaining basic multimodal data and providing data support for the subsequent determination of production multimodal data.

[0068] Basic multimodal data refers to data directly collected by a multimodal data acquisition device, including original image data, sound data, temperature data, and vibration data.

[0069] Step S301: performing data preprocessing on basic multimodal data according to a preset data preprocessing method to generate processed multimodal data.

[0070] Among them, after obtaining the basic multimodal data, the processing terminal performs data preprocessing on the basic multimodal data according to the data preprocessing method, so as to obtain processed multimodal data, providing data support for the subsequent determination of production multimodal data.

[0071] Processing multimodal data refers to multimodal data that has been standardized, synchronized, and feature extracted. Data preprocessing methods refer to methods for preprocessing multimodal data, including standardization: converting data from different sensors into a unified format and scale; time alignment: synchronizing data from all modalities through timestamps to ensure that information at the same time point is represented; feature extraction: extracting effective features from different modal data, such as extracting object contours from image data, extracting audio from sound data, etc. Machine learning or deep learning algorithms can be used to automatically learn effective feature representations from data, such as using convolutional neural networks in image processing and recurrent neural networks or Transformers in natural language processing.

[0072] Step S302: controlling a preset encoder to encode the processed multimodal data to generate encoded multimodal data.

[0073] Among them, after the processing terminal obtains the processed multimodal data, the corresponding encoder is controlled to encode the processed multimodal data, thereby obtaining encoded multimodal data to ensure that data of different modalities can effectively interact and fuse within the same model. The specific encoding method is: for visual data, it is encoded through the visual Transformer, the audio data is encoded through the audio Transformer, and the numerical sensor data is directly encoded through the fully connected layer or other simple linear transformations.

[0074] An encoder is an encoder that encodes multimodal data, including visual transformers, audio transformers, and linear layers. Encoding multimodal data refers to converting multimodal data into a form that can be processed uniformly.

[0075] Step S303: Process the encoded multimodal data according to a preset cross-modal interactive fusion method to generate production multimodal data.

[0076] Among them, the production multimodal data in this step is consistent with the production multimodal data in step S100, and is obtained by the processing terminal after processing the encoded multimodal data according to the cross-modal interactive fusion method.

[0077] The cross-modal interactive fusion method refers to a method of interactively fusing data from different modalities. First, the cross-attention mechanism in the Transformer architecture enables the encoder of each modality to not only focus on its own input, but also refer to the information of other modalities, thereby calculating the similarity scores between them. After normalization by the softmax function, a probability distribution is formed, and the corresponding values ​​are weighted and summed using the probability distribution to obtain a new feature representation. In this way, connections are established between different modalities to enhance the understanding of complex scenes. Finally, fusion is performed at the decision layer, and the advantages of each modality are retained through weighted averaging or voting mechanisms.

[0078] Reference Figure 4 , the steps to analyze the industrial safety production status to determine the production assessment risks include: Step S400: Analyze the industrial production safety status to determine the abnormal status type.

[0079] Among them, the abnormal status type refers to the type of abnormality occurring in the production area, including common abnormalities, such as personnel not wearing protective equipment, high temperature, high noise, equipment failure, etc., and extreme abnormalities, such as explosions and fires, etc., which are obtained by the processing terminal's understanding of the semantics of the industrial safety production status.

[0080] Step S401: Determine whether the abnormal state type meets the requirements of the preset extreme abnormal type.

[0081] Among them, the extreme abnormal type refers to abnormal accidents with greater hazards, such as fire and explosion, and the requirements of the extreme abnormal type refer to being consistent with the extreme abnormal type and stored in the processing terminal by the operator.

[0082] By processing the terminal, it is determined whether the abnormal state type is consistent with the extreme abnormality type, thereby determining whether the abnormality occurring in the production area is an extreme abnormality.

[0083] Step S4011: If it is in compliance, the production assessment risk is determined according to a preset comparison determination method.

[0084] If the processing terminal determines that the abnormal state type is consistent with the extreme abnormal type, it indicates that an extreme abnormality such as a fire or explosion has occurred in the production area. Therefore, the production assessment risk is determined according to the comparison determination method. The specific method is referred to Figure 5 steps.

[0085] The comparative determination method refers to the method used to determine the assessment risk level under extremely abnormal conditions. For specific methods, refer to Figure 5 steps.

[0086] Step S4012: If not, analyze the industrial production safety status to determine the abnormal environment status and abnormal equipment status.

[0087] Among them, if the abnormal state type determined by the processing terminal is inconsistent with the extreme abnormal type, it indicates that a common abnormality has occurred in the production area. Therefore, after analyzing the industrial safety production status, the abnormal environmental state and the abnormal equipment state are determined to provide data support for the subsequent determination of production assessment risks.

[0088] The abnormal environmental state refers to the abnormalities that occur in the production area environment, such as personnel not wearing protective equipment, entering illegal areas, leaving the work area, etc. The abnormal equipment state refers to the abnormalities that occur in the production equipment, such as severe vibration, excessive temperature, etc. Both are obtained by analyzing and classifying the semantics of the industrial safety production status by the processing terminal.

[0089] Step S402: Determine the environmental assessment risk according to the abnormal environmental state and the preset environmental state risk relationship.

[0090] Among them, the environmental status risk relationship refers to the correspondence between the environmental status and the risk level. For example, there is a corresponding risk level when a person does not wear protective equipment, and there is a corresponding risk level when a person enters a violation area. The operator matches the environmental status and the risk level one by one to form a mapping table.

[0091] Environmental assessment risk refers to the risk level of environmental abnormality assessment. After the processing terminal analyzes the environmental abnormality state, it finds the corresponding risk level from the mapping table corresponding to the environmental state risk relationship to obtain the environmental assessment risk. For specific methods, refer to Figure 6 steps.

[0092] Step S403: Determine the equipment assessment risk according to the equipment abnormal state and the preset equipment state risk relationship.

[0093] Among them, the equipment status risk relationship refers to the correspondence between the equipment status and the risk level. For example, equipment temperature rise corresponds to a lower risk level, severe equipment vibration corresponds to a medium risk level, and equipment failure corresponds to a high risk level. The operator matches the equipment status and risk level one by one to form a mapping table.

[0094] The equipment assessment risk refers to the risk of equipment abnormality assessment, which is obtained by searching the mapping table corresponding to the equipment status risk relationship by the processing terminal according to the equipment abnormality status.

[0095] Step S404: Associating the environment assessment risk and the equipment assessment risk to generate a production assessment risk.

[0096] Among them, the production assessment risk in this step is consistent with the production assessment risk in step S1022, and is obtained by associating the environment assessment risk and the equipment assessment risk by the processing terminal. The specific association method is: one association method can simply add the two to obtain the production assessment risk, and another association method can be performed by the operator to set corresponding weights. After calculating the product of the environment assessment risk and the equipment assessment risk with their respective weights, the calculated products are added to obtain the production assessment risk.

[0097] Reference Figure 5 , the steps of determining the production assessment risk according to the preset comparison determination method include: Step S500: Analyze the industrial safety production status to determine the first behavior status type and the extreme abnormal status.

[0098] Among them, the first behavior status type refers to the behavior status of personnel, including whether they have entered the illegal area, whether they have left the work area, whether they have worn protective equipment, etc. The extreme abnormal state refers to the specific extreme state occurring in the production area, including fire or explosion, etc., which is obtained by the processing terminal's understanding of the semantics of the industrial safety production status.

[0099] Step S501: determining whether the first behavior state type meets the requirement of the preset leaving area behavior type.

[0100] Among them, the leaving area behavior type refers to the behavior of the staff leaving the production area, and the leaving area behavior type requirement refers to consistency with the leaving area behavior type.

[0101] By processing the terminal, it is determined whether the first behavior state type is consistent with the leaving area behavior type, so as to determine whether the personnel leave the production area when an extreme abnormality occurs in the production area.

[0102] Step S5011: If it is in compliance, the production assessment risk is determined based on the extreme abnormal state and the preset extreme state risk relationship.

[0103] Among them, if the processing terminal determines that the first behavior state type is consistent with the leaving area behavior type, it means that when extreme abnormalities occur in the production area, the personnel leave the production area. Therefore, the extreme abnormalities occurring in the production area do not cause harm to the personnel. The production assessment risk is determined based on the relationship between the extreme abnormal state and the extreme state risk.

[0104] The extreme state risk relationship refers to the correspondence between the extreme state and the risk level, such as the fire state and the corresponding risk level, the explosion state and the corresponding risk level, etc. The operator forms a mapping table by mapping the extreme state and the risk level one by one. The production assessment risk in this step is consistent with the production assessment risk in step S1022, and is found by the processing terminal in the mapping table corresponding to the extreme state risk relationship according to the extreme abnormal state.

[0105] Step S5012: If not, the preset top-level production risk is defined as the production assessment risk.

[0106] Among them, if the processing terminal determines that the first behavior state type is consistent with the leaving area behavior type, it means that when extreme abnormalities occur in the production area, the personnel did not leave the production area. At this time, the extreme abnormalities can easily cause damage to the personnel, so the top production risk is defined as the production assessment risk.

[0107] Top production risk refers to the highest risk level and is stored by the operator in the processing terminal.

[0108] Reference Figure 6 The steps of determining the environmental assessment risk based on the abnormal environmental state and the preset environmental state risk relationship include: Step S600: Analyze the abnormal state of the environment to determine the second behavior state type.

[0109] Among them, the second behavior status type refers to the behavior of personnel in the production area, including whether the personnel wear protective equipment, whether the personnel enter the illegal area, whether the personnel leave the living area, etc., which is obtained by the processing terminal understanding the semantics of personnel behavior in the abnormal environment state.

[0110] Step S601: Determine whether the second behavior state type meets the requirements of the preset leaving area behavior type.

[0111] The leaving area behavior type and the leaving area behavior type requirement in this step are consistent with the leaving area behavior type and the leaving area behavior type requirement in step S501, and are not described in detail here.

[0112] Whether the second behavior state type is consistent with the requirement of the leaving area behavior type is determined by the processing terminal, so as to determine whether the personnel are still in the production area when a common abnormality occurs in the production area.

[0113] Step S6011: If not, determine the environmental assessment risk according to a preset behavior risk determination method.

[0114] If the processing terminal determines that the second behavior state type is inconsistent with the requirement of leaving the area behavior type, it means that when the ordinary abnormality occurs, the personnel are still in the production area. Therefore, the environmental assessment risk is determined according to the behavior risk determination method. The specific method is referred to Figure 7 steps.

[0115] The behavior risk determination method refers to the method of determining the environmental risk level based on the behavior of personnel, which is stored in the processing terminal by the operator. For specific methods, refer to Figure 7 steps.

[0116] Step S6012: If it is in compliance, the preset unmanned risk is defined as the environmental assessment risk.

[0117] Among them, if the processing terminal determines that the second behavior state type is consistent with the requirements of the leaving area behavior type, it means that when the ordinary abnormality occurs, the personnel are not in the production area. Therefore, the high temperature and high noise in the environmental abnormality will not cause harm to the personnel. Therefore, the unmanned risk is defined as an environmental assessment risk.

[0118] Unmanned risk refers to the risk level of environmental abnormalities when there are no personnel in the production area. It is determined by the operator based on actual conditions and stored in the processing terminal.

[0119] Reference Figure 7 , the steps for determining environmental assessment risks according to the preset behavioral risk determination method include: Step S700: Determine whether the second behavior state type meets the preset requirement of non-dangerous behavior type.

[0120] Among them, the non-dangerous behavior type refers to the behavior of personnel wearing protective equipment neatly, and the requirements of the non-dangerous behavior type refer to the consistency with the non-dangerous behavior type.

[0121] The processing terminal determines whether the second behavior state type is consistent with the non-dangerous behavior type, thereby determining whether the personnel are wearing protective equipment intact when a common abnormality occurs in the environment.

[0122] Step S701: If not, the preset top-level production risk is defined as the environmental assessment risk.

[0123] Among them, if the processing terminal determines that the second behavior state type is inconsistent with the non-dangerous behavior type, it indicates that a common abnormality occurs in the environment, such as high temperature or high noise, and the personnel do not wear protective equipment properly. At this time, the common abnormality will cause damage to the personnel, so the top production risk is defined as the environmental assessment risk. The top production risk in this step is consistent with the top production risk in step S5012, and will not be repeated here.

[0124] Step S702: If it is in compliance, the environmental assessment risk is determined based on the environmental abnormality state and the environmental state risk relationship.

[0125] Among them, if the processing terminal determines that the second behavior state type is consistent with the non-dangerous behavior type, it indicates that a common abnormality has occurred in the environment. For example, when the temperature is high or the noise is high, the personnel are wearing protective equipment intact. At this time, the abnormal environment causes less harm to the personnel. Therefore, according to the abnormal state of the environment, the environmental assessment risk corresponding to the current abnormality is found in the mapping table corresponding to the environmental state risk relationship.

[0126] Based on the same inventive concept, the embodiment of the present application provides an industrial production monitoring system based on a large model, including: An acquisition module is used to acquire production multimodal data, area usage serial numbers, and equipment usage serial numbers; A memory for storing a program of an industrial production monitoring method based on a large model; The program in the processor memory can be loaded and executed by the processor to implement the industrial production monitoring method based on the large model.

[0127] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0128] An embodiment of the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute an industrial production monitoring method based on a large model.

[0129] Computer storage media include, for example, various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0130] Based on the same inventive concept, an embodiment of the present application provides an intelligent terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes an industrial production monitoring method based on a large model.

[0131] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0132] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. An industrial production monitoring method based on a large model, characterized in that: include: Obtain production multimodal data of the preset production area; Generate industrial safety production status by inference based on the preset industrial safety monitoring model of production multimodal data control; Determine whether the industrial safety production status meets the requirements of the preset normal production status; If it meets the requirements, continue to obtain the multimodal production data of the production area for cyclic judgment; If not, the industrial safety production status is analyzed to determine the production assessment risk; Determine real-time alarm data based on production assessment risk, industrial safety production status and preset risk alarm relationships; Provide alarm prompts based on real-time alarm data.

2. The industrial production monitoring method based on a large model according to claim 1 is characterized in that: The steps of obtaining production multimodal data of a preset production area include: Obtain the area usage serial number and equipment usage serial number of the production area; Determine the starting area device serial number according to the area use serial number and the preset area serial number device relationship; Determine the device serial number of the startup device according to the device usage serial number and the preset device serial number device relationship; According to the device serial number of the starting area and the device serial number of the starting equipment, the preset multimodal data acquisition device is controlled to collect the production data of the production area to generate production multimodal data.

3. The industrial production monitoring method based on a large model according to claim 2 is characterized in that: The step of controlling a preset multimodal data acquisition device to collect production data of a production area according to the device serial number of the start-up area and the device serial number of the start-up equipment to generate production multimodal data comprises: Controlling the multimodal data acquisition device to collect production data of the production area according to the device serial number of the start-up area and the device serial number of the start-up equipment to generate basic multimodal data; Performing data preprocessing on the basic multimodal data according to a preset data preprocessing method to generate processed multimodal data; Controlling a preset encoder to encode the processed multimodal data to generate encoded multimodal data; The encoded multimodal data is processed according to a preset cross-modal interactive fusion method to generate production multimodal data.

4. The industrial production monitoring method based on a large model according to claim 1 is characterized in that: The steps to analyze the industrial safety production status to determine the production assessment risks include: Analyze the industrial safety production status to determine the type of abnormal status; Determine whether the abnormal state type meets the requirements of the preset extreme abnormal type; If it meets the requirements, the production assessment risk is determined according to the preset comparison determination method; If not, analyze the industrial safety production status to determine the abnormal environmental status and equipment status; Determine environmental assessment risks based on the relationship between environmental abnormalities and preset environmental status risks; Determine the equipment assessment risk based on the equipment abnormal status and the preset equipment status risk relationship; Associate environmental assessment risks and equipment assessment risks to generate production assessment risks.

5. The industrial production monitoring method based on a large model according to claim 4 is characterized in that: The steps for determining production assessment risks according to the preset comparative determination method include: Analyze the industrial safety production status to determine the first behavior status type and extreme abnormal status; Determine whether the first behavior state type meets the requirements of the preset leaving area behavior type; If it is in compliance, the production assessment risk is determined based on the extreme abnormal state and the preset extreme state risk relationship; If not met, the preset top production risk is defined as the production assessment risk.

6. The industrial production monitoring method based on a large model according to claim 4 is characterized in that: The steps for determining environmental assessment risks based on the relationship between environmental abnormalities and preset environmental status risks include: Analyze the abnormal state of the environment to determine the type of the second behavior state; Determine whether the second behavior state type meets the requirements of the preset leaving area behavior type; If not, the environmental assessment risk is determined based on the preset behavioral risk determination method; If it meets the requirements, the preset unmanned risk is defined as the environmental assessment risk.

7. The industrial production monitoring method based on a large model according to claim 6 is characterized in that: The steps to determine environmental assessment risks according to the preset behavioral risk determination method include: Determining whether the second behavior state type meets the requirements of the preset non-dangerous behavior type; If not, the preset top production risk is defined as the environmental assessment risk; If it meets the requirements, the environmental assessment risk will be determined based on the relationship between the abnormal environmental state and the environmental state risk.

8. The industrial production monitoring system based on the large model is characterized by: include: An acquisition module, used to acquire production multimodal data; A memory for storing a program of the industrial production monitoring method based on a large model according to any one of claims 1 to 7; The program in the memory can be loaded and executed by the processor to implement the large model-based industrial production monitoring method as described in any one of claims 1 to 7.

9. An intelligent terminal, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes the large model-based industrial production monitoring method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program is stored which can be loaded by a processor and execute the large model-based industrial production monitoring method according to any one of claims 1 to 7.

Images