Potential safety hazard inspection method based on large model

Abstract

The invention discloses a potential safety hazard inspection method based on a large model, and belongs to the technical field of production safety, and the method comprises the steps: building a cloud large model at a cloud; processing the cloud large model according to the edge end information to obtain an edge large model suitable for the edge end; configuring the edge large model in the edge end; the method comprises the following steps: acquiring multi-modal acquisition data in a safety production scene, processing the multi-modal acquisition data according to a preset processing mode, and marking the processed multi-modal acquisition data as multi-modal data; an analysis and prediction model is configured for an edge end according to an area where a user is located, real-time analysis is carried out through the analysis and prediction model, corresponding prediction complexity is obtained, an edge large model adjusts corresponding reasoning precision according to the prediction complexity, and safety analysis is carried out on multi-modal data according to the edge large model with the corresponding reasoning precision. And obtaining a corresponding potential safety hazard inspection result.

Description

Technical Field

[0001] This invention belongs to the field of production safety technology, specifically a method for inspecting safety hazards based on a large model. Background Technology

[0002] Safe production is the core prerequisite for high-quality development in all industries, and timely identification and handling of safety hazards are key means to reduce the incidence of production safety accidents. Traditional methods of safety hazard inspection mainly rely on manual inspections and single sensor monitoring, which have problems such as low identification efficiency, high missed detection rate, and weak cross-modal data fusion capabilities.

[0003] With the development of artificial intelligence technology, deep learning models have begun to be applied to safety hazard identification. However, existing models are mostly designed for analyzing single-modal data (such as images or text only), which makes it difficult to handle the multi-source heterogeneous data features in safety production scenarios. At the same time, large-scale pre-trained models usually rely on cloud computing power, which has problems such as high data transmission latency and high bandwidth costs, and cannot meet the real-time requirements of edge inspection equipment.

[0004] Therefore, developing a safety hazard inspection method that can integrate multimodal data and support real-time safety identification at the edge has become an urgent technical problem to be solved in the field of production safety. Summary of the Invention

[0005] To address the problems of the above solutions, this invention provides a safety hazard inspection method based on a large model.

[0006] The objective of this invention can be achieved through the following technical solutions: A safety hazard inspection method based on a large model, the method includes: Step 1: Build a large cloud model in the cloud; process the large cloud model based on the edge information to obtain an edge model suitable for the edge; configure the edge model in the edge.

[0007] Step 2: Acquire multimodal data in a safe production scenario, process the multimodal data according to a preset processing method, and mark the processed multimodal data as multimodal data.

[0008] Step 3: Configure an analysis and prediction model for the edge device based on the user's location. The analysis and prediction model is used to predict the complexity of multimodal data acquisition. Furthermore, the configuration methods for the predictive model are analyzed, including: The target region is determined based on the user's location, and each collaborating user is determined based on the target region. The platform initializes a global model based on each collaborating user and the user, and distributes a copy of the global model to each collaborating user and the user. Collaborating users and individual users train a copy of the global model locally using local data to obtain the corresponding model parameters, which are then sent to the platform. The global model is updated based on these parameters, and the updated global model undergoes lightweight processing to obtain the analysis and prediction models for each collaborating user and individual user. These analysis and prediction models are then configured at the edge.

[0009] Furthermore, the methods for determining the target area include: Based on the preset regional feature items, feature recognition is performed on each unit region within the platform's service area to obtain the unit feature set of each unit region. Feature evaluation is performed on the feature set of each unit to obtain the regional evaluation results between the regions of each unit. The regional evaluation results include regions that are the same and regions that are different. Unit regions with the same regional assessment result are grouped into one category to obtain several regional classifications. Within each regional classification, the regional assessment results of each unit region are the same. Identify the unit area where the user is located, match the corresponding area classification based on the unit area, and determine the target area based on the area classification.

[0010] Furthermore, feature evaluation is performed on the feature sets of each unit, including: A regional assessment model is established, and its expression is as follows: ; In the formula: (q, p) are the input data, q and p are two unit feature sets respectively, q↔p means that the two unit feature sets meet the classification criteria; the output data is the region evaluation value QP(q, p), and the region evaluation value is 1 or 0; The feature sets of the two corresponding units are integrated into the input data and input into the regional assessment model for analysis to obtain the corresponding regional assessment value QP(q, p); When the regional assessment value is 1, the regional assessment result is that the regions are the same; When the regional assessment value is 0, the regional assessment result is that the regions are not the same.

[0011] Furthermore, based on the target area, each collaborative user is identified, including: Real-time identification of potential collaborative users within the target area; collaborative analysis of each potential collaborative user and user; obtaining collaborative evaluation results between each potential collaborative user and user, including collaborative success and collaborative failure. Potential collaborative users whose collaborative assessment results are qualified are marked as collaborative users.

[0012] Furthermore, collaborative analysis is conducted on each potential collaborating user and the existing user, including: Obtain local data information for each potential collaborative user and user, and label the local data information as potential collaborative data information and basic data information according to the potential collaborative user and user respectively; The potential collaborative data information is simulated and analyzed with the basic data information to obtain the predictive simulation effect of using the potential collaborative data information as collaborative data information; the predictive simulation effect is compared with the preset collaborative conditions to obtain the collaborative evaluation result.

[0013] Furthermore, once the analysis and prediction model has been deployed at the user's edge, the collaborating users within the target area are identified, allowing them to participate in the collaborative optimization of the analysis and prediction model.

[0014] Step 4: Perform real-time analysis using the aforementioned analysis and prediction model to obtain the corresponding prediction complexity. The edge big model adjusts the corresponding inference accuracy based on the prediction complexity. Based on the corresponding inference accuracy, the edge big model performs security analysis on the multimodal data to obtain the corresponding security hazard inspection results.

[0015] Furthermore, the standard complexity of multimodal data is identified in real time, and the prediction complexity corresponding to the multimodal data is obtained. Based on the standard complexity and prediction complexity of the multimodal data, corresponding optimized data is generated, and the analysis and prediction model is optimized and adjusted based on the optimized data.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The safety hazard inspection method proposed in this invention effectively overcomes the limitations of traditional methods that rely solely on single-modal data through an innovative multimodal data fusion mechanism. It comprehensively captures the feature information of multi-source heterogeneous data such as images and text in safety production scenarios, significantly improving the accuracy and comprehensiveness of safety hazard identification and reducing the false negative rate. Simultaneously, addressing the problem of excessive reliance on cloud computing power in existing deep learning models, this invention optimizes the model architecture and algorithm design, enabling it to run efficiently on edge devices without transmitting large amounts of data to the cloud. This not only significantly reduces data transmission latency and bandwidth costs but also achieves real-time identification and rapid response to safety hazards, providing a more reliable and efficient technical guarantee for safe production and powerfully promoting the high-quality development of safe production towards intelligence and real-time capabilities. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, a safety hazard inspection method based on a large model includes: Step 1: Build a large-scale model for security hazard inspection in the cloud, and mark it as the cloud-based large-scale model; the cloud-based large-scale model is used to analyze multimodal data and identify corresponding security hazards; based on knowledge distillation and quantization technology, construct a lightweight large-scale model branch suitable for edge inspection equipment, transfer the core knowledge of the cloud-based large-scale model to the lightweight model, and obtain the edge large-scale model; configure the edge large-scale model at the edge.

[0021] In one embodiment, the cloud-based big model is built upon existing big model technologies.

[0022] In one embodiment, the edge large model has multiple different inference accuracies. Therefore, depending on the differences in actual edge inspection equipment, multiple edge large models with different inference accuracies can be set at the edge, or a single edge large model can be set but with multiple inference accuracies. The specific choice is made based on the actual situation such as the timeliness of hazard identification and edge resources.

[0023] In one embodiment, the edge device can be various related devices such as inspection robots.

[0024] Step 2: Acquire multimodal data in a safe production scenario, process the multimodal data according to a preset processing method, and mark the processed multimodal data as multimodal data.

[0025] In one embodiment, the preset processing method is the existing preprocessing of the collected data and other related data.

[0026] For example, the multimodal data includes at least two or more of the following: image data, video data, text ledger data, and sensor time-series data. Differentiated preprocessing operations are performed on different types of data, including scene normalization processing for image data to eliminate the influence of environmental factors such as lighting and angle on hazard identification; structured entity extraction processing is performed on text ledger data to extract information such as keywords related to hazards, responsible entities, and rectification records; and outlier filtering processing is performed on sensor time-series data to remove interference from noisy data on the analysis results.

[0027] Step 3: Configure an analysis and prediction model for the edge device based on the user's location. The analysis and prediction model is used to predict the complexity of multimodal data collection, so as to switch the appropriate large model inference accuracy according to the complexity.

[0028] In one embodiment, the analytical prediction model can be based on existing prediction techniques and analysis of relevant historical data at the user's location.

[0029] In one embodiment, due to changes in production workers, managers, climate, equipment, etc., building an analytical prediction model solely based on historical data from the user has certain limitations. Therefore, in this embodiment, the analytical prediction model is configured using the following method: Identifying the user's location refers to the area where the production plant or enterprise is located, such as a town, county, or district, and marking it as the target area. Based on the target area, each collaborating user is identified. The platform initializes a global model for each collaborating user and individual user, and distributes copies of the global model to each collaborating user and individual user. The collaborating users and individual users train the global model copy locally using local data to obtain the corresponding model parameters, which are then sent to the platform. The global model is updated based on the model parameters, and the updated global model undergoes lightweight processing, such as pruning, quantization, and knowledge distillation, to obtain the analysis and prediction models for each collaborating user and individual user. The analysis and prediction models can be either a unified lightweight global model or personalized analysis and prediction models based on the different needs of users or collaborating users. These analysis and prediction models are configured at the edge.

[0030] The establishment and training of the specific global model are implemented based on existing federated learning techniques; it is used to predict the complexity of subsequent multimodal acquisition data based on the current multimodal acquisition data.

[0031] Complexity can be quantified using various metrics, such as image resolution, noise level, number of targets, scene complexity, audio signal-to-noise ratio, frequency range, and text data vocabulary size, sentence length, semantic complexity, etc.

[0032] In one embodiment, dividing the target area solely by town, county, etc., may result in a limited number of selectable collaborative users. Therefore, the target area is determined based on regional characteristics. The method for identifying the target area includes: The platform summarizes various regional feature items that affect the complexity of data collection due to regional differences based on historical data. This mainly focuses on the impact of regional differences in personnel and equipment. For example, the capabilities of frontline workers vary in different regions. When new workers enter a user's factory, the lack of relevant data leads to differences in safety hazards in their work. Therefore, it is necessary to improve the accuracy of the analysis in a targeted manner. That is, the complexity in this embodiment not only focuses on the complexity of the data, but also on the complexity of differences in personnel and equipment. The platform can manually set the regional feature items to determine whether different regions can be considered the same. Based on the feature items of each region, feature identification is performed on each unit region within the service area of ​​the platform to obtain the unit feature set of each unit region. The unit region can be set to the smallest region such as village or town to facilitate the rapid identification of each unit region. Feature evaluation is performed on the feature set of each unit to obtain the regional evaluation results between the regions of each unit. The regional evaluation results include regions that are the same and regions that are different. Unit regions with the same regional assessment result are grouped into one category, resulting in several regional classifications; within each regional classification, the regional assessment results of all unit regions are the same. Identify the unit area where the user is located, match the corresponding region category according to the unit area, and mark the region corresponding to the region category as the target region. That is, each unit area corresponding to the region category belongs to the target region.

[0033] In one embodiment, feature evaluation is performed on each unit feature set. The evaluation criterion is whether they belong to the same category. The classification criterion is that two unit feature sets can be considered the same, or they can be determined by a similarity greater than a preset value. Alternatively, the platform can label the corresponding training set based on historical data and build an intelligent model based on machine learning, deep learning algorithms, etc. Because regional features do not change frequently, evaluation can also be performed manually to obtain regional evaluation results.

[0034] In one embodiment, feature evaluation is performed on the feature sets of each unit, including: A regional assessment model is established, and its expression is as follows: ; In the formula: (q, p) are the input data, q and p are two unit feature sets respectively, q↔p means that the two unit feature sets meet the classification criteria. The platform marks which unit feature sets meet the classification criteria and which do not. Those that do not meet the classification criteria are regarded as abnormal data and are used to form a training set for training. The output data is the region evaluation value QP(q, p), and the region evaluation value is 1 or 0. The feature sets of the two corresponding units are integrated into the input data and input into the regional assessment model for analysis to obtain the corresponding regional assessment value QP(q, p); When the regional assessment value is 1, the regional assessment result is that the regions are the same; When the regional assessment value is 0, the regional assessment result is that the regions are not the same.

[0035] In one embodiment, determining each collaborative user based on the target region includes: Other users within the target area who are not this user are marked as potential collaborating users. Potential collaborating users within the target area are identified in real time. Collaboration analysis is performed on each potential collaborating user and the user to determine whether potential collaborating users and the user can be trained collaboratively. This improves the prediction accuracy of the analysis and prediction model and obtains the corresponding collaboration evaluation results, which include collaboration qualified and collaboration unqualified. Potential collaborative users whose collaborative assessment results are qualified are marked as collaborative users.

[0036] In one embodiment, collaborative analysis is performed on each potential collaborative user and the user to analyze the effect of the potential collaborative user's local data on the improvement of the prediction accuracy of the analysis and prediction model. This analysis can be based on historical training effect data, and potential collaborative users whose improvement effect is greater than the collaborative condition are considered to be qualified for collaboration. The collaborative condition can be set to simply have an improvement in effect, which is set by the platform.

[0037] Based on the above approach, various existing technologies can be used to achieve collaborative analysis, such as collaborative analysis based on machine learning.

[0038] For example, collaborative analysis is performed on each potential collaborating user and the user, including: Acquire local data information of each potential collaborating user and user, i.e., what local data they possess. This can be obtained by users and potential collaborating users uploading the corresponding local data information, or by the platform analyzing and determining based on user information, etc.; mark the local data information as potential collaborating data information and basic data information according to potential collaborating users and users respectively; The potential collaborative data information is simulated and analyzed with the basic data information to obtain the predictive simulation effect of using the potential collaborative data information as collaborative data information; that is, to estimate the simulation effect such as prediction accuracy and modeling efficiency when the potential collaborative data information is used as collaborative data information in model training, and to determine the estimated simulation effect using existing technologies, such as statistical determination based on historical training effect data; the predicted simulation effect is compared with the preset collaborative conditions to obtain the collaborative evaluation result.

[0039] In one embodiment, when the user's edge device has already deployed the analysis and prediction model, the collaborative users are also identified in real time. The platform then deploys the corresponding analysis and prediction model or a copy of the global model for the newly identified collaborative users, making it easier for the new collaborative users to use the prediction. Subsequently, the model is learned and optimized based on the data generated by the user and each collaborative user during use, forming new model parameters. The global model is then optimized using the new model parameters to achieve continuous optimization.

[0040] Step 4: Perform real-time analysis through the prediction model to obtain the corresponding prediction complexity. To distinguish it from the complexity of the currently collected multimodal data, mark it as the prediction complexity. The edge big model adjusts the corresponding inference accuracy according to the prediction complexity. Based on the corresponding inference accuracy, the edge big model performs security analysis on the multimodal data to obtain the corresponding security hazard inspection results.

[0041] In one embodiment, in order to improve the accuracy of the analysis and prediction model, optimization and adjustment are performed based on the complexity of the actual multimodal data and the prediction complexity, that is, it can perform self-learning based on the result data.

[0042] For example, the actual complexity of multimodal data is identified in real time and marked as standard complexity. The predicted complexity corresponding to the multimodal data is obtained. Based on the standard complexity and predicted complexity of the multimodal data, corresponding optimized data is formed. The analysis and prediction model is optimized and adjusted based on the optimized data.

[0043] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a security vulnerability inspection method based on a large model as described in the above embodiments.

[0044] The above formulas are all numerical calculations after removing dimensions. The formulas are obtained by software simulation based on a large amount of data and are closest to the real situation. The preset parameters and preset thresholds in the formulas are set by those skilled in the art according to the actual situation or obtained by simulation based on a large amount of data.

[0045] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for safety hazard inspection based on a large model, characterized in that, The methods include: Step 1: Build a large cloud model; The cloud-based large model is processed based on edge information to obtain an edge large model suitable for the edge. Configure the large edge model at the edge end; Step 2: Acquire multimodal data in a safe production scenario, process the multimodal data according to a preset processing method, and mark the processed multimodal data as multimodal data; Step 3: Configure an analysis and prediction model for the edge device based on the user's location. The analysis and prediction model is used to predict the complexity of multimodal data acquisition. Step 4: Perform real-time analysis using the aforementioned analysis and prediction model to obtain the corresponding prediction complexity. The edge big model adjusts the corresponding inference accuracy based on the prediction complexity. Based on the corresponding inference accuracy, the edge big model performs security analysis on the multimodal data to obtain the corresponding security hazard inspection results.

2. The safety hazard inspection method based on a large model according to claim 1, characterized in that, The configuration methods for the analysis and prediction model include: The target region is determined based on the user's location, and each collaborating user is determined based on the target region. The platform initializes a global model based on each collaborating user and the user, and distributes a copy of the global model to each collaborating user and the user. Collaborating users and individual users train a copy of the global model locally using local data to obtain the corresponding model parameters, which are then sent to the platform. The global model is updated based on these parameters, and the updated global model undergoes lightweight processing to obtain the analysis and prediction models for each collaborating user and individual user. These analysis and prediction models are then configured at the edge.

3. The safety hazard inspection method based on a large model according to claim 2, characterized in that, Methods for determining the target region include: Based on the preset regional feature items, feature recognition is performed on each unit region within the platform's service area to obtain the unit feature set of each unit region. Feature evaluation is performed on the feature set of each unit to obtain the regional evaluation results between the regions of each unit. The regional evaluation results include regions that are the same and regions that are different. Unit regions with the same regional assessment result are grouped into one category to obtain several regional classifications. Within each regional classification, the regional assessment results of each unit region are the same. Identify the unit area where the user is located, match the corresponding area classification based on the unit area, and determine the target area based on the area classification.

4. The safety hazard inspection method based on a large model according to claim 3, characterized in that, Feature evaluation is performed on the feature sets of each unit, including: A regional assessment model is established, and its expression is as follows: ； In the formula: (q, p) are the input data, q and p are two unit feature sets respectively, q↔p means that the two unit feature sets meet the classification criteria; the output data is the region evaluation value QP(q, p), and the region evaluation value is 1 or 0; The feature sets of the two corresponding units are integrated into the input data and input into the regional assessment model for analysis to obtain the corresponding regional assessment value QP(q, p); When the regional assessment value is 1, the regional assessment result is that the regions are the same; When the regional assessment value is 0, the regional assessment result is that the regions are not the same.

5. The safety hazard inspection method based on a large model according to claim 2, characterized in that, Identify the collaborative users based on the target region, including: Real-time identification of potential collaborative users within the target area; collaborative analysis of each potential collaborative user and user; obtaining collaborative evaluation results between each potential collaborative user and user, including collaborative success and collaborative failure. Potential collaborative users whose collaborative assessment results are qualified are marked as collaborative users.

6. The safety hazard inspection method based on a large model according to claim 5, characterized in that, Collaboration analysis is conducted on each potential collaborating user and the existing user, including: Obtain local data information for each potential collaborative user and user, and label the local data information as potential collaborative data information and basic data information according to the potential collaborative user and user respectively; The potential collaborative data information is simulated and analyzed with the basic data information to obtain the predictive simulation effect of using the potential collaborative data information as collaborative data information; the predictive simulation effect is compared with the preset collaborative conditions to obtain the collaborative evaluation result.

7. The safety hazard inspection method based on a large model according to claim 1, characterized in that, Once the analysis and prediction model has been deployed at the user's edge, continue to identify collaborating users within the target area and allow them to participate in the collaborative optimization of the analysis and prediction model.

8. The safety hazard inspection method based on a large model according to claim 1, characterized in that, Real-time identification of the standard complexity of multimodal data, and acquisition of the corresponding prediction complexity of multimodal data; Based on the standard complexity and prediction complexity of the multimodal data, corresponding optimized data is generated, and the analysis and prediction model is optimized and adjusted based on the optimized data.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a security hazard inspection method based on a large model as described in any one of claims 1 to 8.

Images