Safety monitoring method based on mixing of large model and neural network algorithm

Abstract

The embodiment of the invention provides a safety monitoring method based on mixing of a large model and a neural network algorithm, and the method comprises the steps: carrying out the real-time detection through deploying a lightweight convolutional neural network at an edge end, and screening a to-be-rechecked sample through multi-scale feature extraction and a confidence scoring mechanism; a deep neural network model is deployed at a cloud end to execute fine analysis, deep semantic features are extracted by using a residual structure and an attention mechanism, and a re-check decision is performed based on feature sensitivity and a region weight. Meanwhile, a double-cache structure is constructed to maintain a training sample pool, differential updating strategies are achieved according to sample importance, model performance is continuously optimized through knowledge distillation and comparative learning, edge computing resource constraints and detection precision requirements are effectively balanced, and intelligent upgrading of a safety monitoring system is achieved; according to the method, the detection accuracy and the real-time response capability of the system can be improved by constructing an intelligent framework of edge cloud collaboration and combining the advantages of a large model and a lightweight neural network.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a safety monitoring method based on a hybrid of a large model and a neural network algorithm. Background Art

[0002] With the in-depth development of industrial intelligence, the application of safety monitoring systems in production sites is becoming increasingly popular. Traditional video surveillance solutions mainly rely on manual inspections, which have problems such as poor real-time performance and high missed detection rates. Intelligent monitoring solutions that rely solely on edge computing or cloud computing often find it difficult to achieve a good balance between resource consumption and detection accuracy.

[0003] The current mainstream security monitoring systems have obvious limitations in model deployment and optimization. Edge models often sacrifice detection accuracy in pursuit of lightweight, while cloud models have higher accuracy but larger response delays. This split architectural design makes it difficult to fully leverage the advantages of edge-cloud collaboration and cannot effectively respond to complex and changing on-site security monitoring needs. At the same time, the existing system is insufficient in processing model updates and knowledge migration, making it difficult to continuously improve detection results.

[0004] In terms of sample management and model optimization, the existing technology still has a lot of room for improvement. The system lacks an effective sample screening mechanism and training strategy, making it difficult to extract valuable training samples from massive data. In addition, the lack of adaptive optimization capabilities during the model update process affects the continuous evolution of the system. These problems restrict the practicality and reliability of the security monitoring system. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides a security monitoring method based on a hybrid of large models and neural network algorithms. It can improve the system's detection accuracy and real-time response capabilities by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a safety monitoring method based on a hybrid of a large model and a neural network algorithm, comprising: An edge safety detection model is constructed based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted through a combination of a deep convolutional layer and a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on edge devices for real-time data collection and reasoning. Target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized features are transmitted to the cloud for review. A deep neural network model is deployed in the cloud as a review system, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, and calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with time series correlation analysis, and updates the alarm status based on the review results and feeds back the review results to the edge; A double cache structure is constructed to maintain a training sample pool, wherein the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map are combined into training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for periodic batch training, a small batch of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0007] Furthermore, the edge safety detection model is constructed based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer with a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features, including: The lightweight convolutional neural network is divided into a feature extraction part and a feature fusion part. The feature extraction part uses 5 layers of depth-separable convolutional layers for feature extraction. The convolution kernel size of each depth-separable convolutional layer is 3×3, the number of channels is 32, 64, 128, 256, and 512, respectively, and the step size is 2. After each depth-separable convolutional layer, a batch normalization layer is connected to standardize the feature data to obtain the multi-scale features; Three bottleneck layer modules are set in the feature fusion part, each of the bottleneck layer modules includes a series structure of a 1×1 convolution layer, a 3×3 deep convolution layer and a 1×1 convolution layer, and the number of output channels of the bottleneck layer modules is 256, 128, and 64, respectively. The multi-scale features are input into the bottleneck layer module, and the output of the bottleneck layer module is nonlinearly transformed by using a rectified linear unit activation function to obtain the optimized features.

[0008] Furthermore, the security detection model is deployed on the edge device to perform real-time data collection and reasoning, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores, including: The security detection model is quantized and compressed, the model weight is converted from a floating point number to an 8-bit integer, the security detection model is divided into 32 computing units, the computing units are deployed in the neural network processor of the edge device in a parallel computing manner, and video data with a resolution of 1920×1080 is collected based on the image sensor of the edge device, and the video data is parsed into image frames at a frequency of 30 frames per second to obtain the real-time collected data; The real-time collected data is input into the security detection model, a three-level feature pyramid network is constructed through the optimized features, a candidate area is generated by an anchor box prediction method, non-maximum suppression is performed on the candidate area to remove overlapping boxes, the category probability distribution of the candidate area is calculated to obtain the detection result, and the category probability distribution is normalized using a softmax function to obtain the confidence score.

[0009] Furthermore, the deep neural network model is deployed in the cloud as a review system, the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis of the detection results based on the enhanced features to obtain review results, including: Constructing a 50-layer residual neural network, wherein the residual neural network includes four residual module groups, each of which is composed of a plurality of residual units, wherein the residual unit adopts a serial structure of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer, and the number of output channels of the residual module group is 256, 512, 1024, and 2048, respectively, and inputting the optimized features into the residual neural network for feature transformation to obtain the deep semantic features; A channel attention branch and a spatial attention branch are constructed, wherein the channel attention branch uses a global average pooling layer and a fully connected layer to generate a channel weight, and the spatial attention branch uses a convolutional layer and a maximum pooling layer to generate a spatial weight. The channel weight and the spatial weight are respectively element-wise multiplied with the deep semantic features to obtain the enhanced features, and the enhanced features are used to correct the target bounding box and the category label of the detection result to obtain the review result.

[0010] Further, the activation response of the enhanced feature at each layer of the edge model is calculated to obtain a feature sensitivity map, the spatial attention weight of the key area is extracted based on the feature sensitivity map to obtain a regional weight map, the sample importance score is generated by combining the regional weight map with the time series correlation analysis, the alarm state is updated based on the review result and the review result is fed back to the edge, including: Input the enhanced feature into the edge security detection model, record the feature map of each convolutional layer in the edge model, calculate the gradient value of the enhanced feature to the feature map of each layer by back propagation, perform global average pooling and L2 normalization on the gradient value, and perform weighted summation of the normalized gradient value and the feature map of the corresponding layer to obtain the feature sensitivity map; Perform adaptive threshold segmentation on the feature sensitivity map to extract high response areas, calculate the spatial position and area ratio of the high response area in the original image to obtain the regional weight map, construct a long short-term memory network to perform time series analysis on the regional weight map in 10 consecutive frames of video, perform weighted fusion of the time series analysis results and the spatial statistical features of the regional weight map to obtain the sample importance score, perform credibility assessment on the review results based on the sample importance score, feed back the review results to the edge end through a message queue and update the alarm status.

[0011] Further, the dual buffer structure is constructed to maintain a training sample pool, the training sample pool includes an online buffer area and an offline storage area, the enhanced features, the review results and the regional weight map form a training sample, the training samples are prioritized based on the sample importance scores, the training samples with higher priority are stored in the online buffer area for immediate update, and the training samples with lower priority are stored in the offline storage area for regular batch training, including: A sample database based on key-value pair storage is established in the cloud server, the online cache area is set as a high-speed memory database, the storage capacity of the online cache area is 1000 training samples, the offline storage area is set as a solid-state hard disk database, the storage capacity of the offline storage area is 10000 training samples, a unique identifier is assigned to each of the training samples, and the enhanced features, the review results and the regional weight map are serialized and stored in association with the identifier; The sample importance score is normalized to the range of 0 to 1, and the sample importance threshold is set to 0.8. The training samples are arranged in descending order according to the sample importance score, and the training samples with scores greater than the sample importance threshold are written into the online cache area. When the number of samples in the online cache area exceeds the storage capacity, the sample with the lowest score is deleted, and the training samples with scores less than the sample importance threshold are written into the offline storage area. The samples in the offline storage area are batch-trained and updated every 24 hours.

[0012] Furthermore, in each round of updating, a small batch of samples is extracted from the online cache area to construct a training set, knowledge distillation is performed using the training set to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update result to perform system adaptive optimization, including: A stratified sampling strategy is adopted to obtain training samples from the online cache area, each small batch contains 32 training samples, the stratified sampling is balanced based on the sample distribution of three types of tasks: helmet wearing, reflective clothing wearing and personnel behavior, the enhanced features of the training samples are input into the teacher model to obtain soft labels, and the soft labels are paired with the review results to construct the training set; The training set is input into the student model to obtain a prediction result, the cross entropy between the prediction result and the soft label is calculated to obtain the distillation loss, a feature contrast loss function is constructed to measure the similarity between the output features of the student model and the teacher model, a weighted sum of the distillation loss and the feature contrast loss is used as the total loss function, the total loss function is optimized using the stochastic gradient descent method to update the student model parameters, the learning rate is adaptively adjusted based on the rate of change of the total loss function, and the confidence threshold is dynamically updated according to the accuracy on the validation set.

[0013] In a second aspect, the present application provides a safety monitoring device based on a hybrid of a large model and a neural network algorithm, comprising: A confidence scoring module is used to build an edge safety detection model based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolution layer and a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device for real-time data collection and reasoning. Target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized features are transmitted to the cloud for review. A review alarm module is used to deploy a deep neural network model in the cloud as a review system, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, performs a refined analysis on the detection results based on the enhanced features to obtain a review result, calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, combines the regional weight map with time series correlation analysis to generate a sample importance score, updates the alarm status based on the review result and feeds the review result back to the edge; A dynamic update module is used to build a double cache structure to maintain a training sample pool, wherein the training sample pool includes an online cache area and an offline storage area, wherein the enhanced features, the review results and the regional weight map form training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, small batches of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0014] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the security monitoring method based on a hybrid of a large model and a neural network algorithm are implemented.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the security monitoring method based on a hybrid of a large model and a neural network algorithm.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the security monitoring method based on a hybrid of a large model and a neural network algorithm.

[0017] It can be seen from the above technical solutions that the present application provides a security monitoring method based on a hybrid of large models and neural network algorithms. It performs real-time detection by deploying lightweight convolutional neural networks at the edge, and screens samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploys deep neural network models in the cloud to perform refined analysis, extracts deep semantic features using residual structures and attention mechanisms, and makes review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system. The present application can improve the detection accuracy and real-time response capabilities of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the safety monitoring method based on the hybrid of large model and neural network algorithm in the embodiment of the present application; Figure 2 This is the second flow chart of the safety monitoring method based on the hybrid of large model and neural network algorithm in the embodiment of the present application; Figure 3 This is a third flow chart of a safety monitoring method based on a hybrid of a large model and a neural network algorithm in an embodiment of the present application; Figure 4 This is a fourth flow chart of a safety monitoring method based on a hybrid of a large model and a neural network algorithm in an embodiment of the present application; Figure 5This is a fifth flow chart of a safety monitoring method based on a hybrid of a large model and a neural network algorithm in an embodiment of the present application; Figure 6 This is the sixth flow chart of the safety monitoring method based on the hybrid of large model and neural network algorithm in the embodiment of the present application; Figure 7 FIG7 is a flow chart of a safety monitoring method based on a hybrid of a large model and a neural network algorithm in an embodiment of the present application; Figure 8 A structural diagram of a safety monitoring device based on a hybrid of a large model and a neural network algorithm in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0020] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0023] Taking into account the problems existing in the prior art, this application provides a security monitoring method based on a hybrid of large models and neural network algorithms. It performs real-time detection by deploying lightweight convolutional neural networks at the edge, and screens samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms. It deploys deep neural network models in the cloud to perform refined analysis, extracts deep semantic features using residual structures and attention mechanisms, and makes review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented based on the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system. This application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0024] In order to improve the detection accuracy and real-time response capability of the system by building an intelligent framework of edge-cloud collaboration and combining the advantages of large models and lightweight neural networks, this application provides an embodiment of a security monitoring method based on a hybrid of large models and neural network algorithms, see Figure 1 The safety monitoring method based on the hybrid of large model and neural network algorithm specifically includes the following contents: Step S101: construct an edge safety detection model based on a lightweight convolutional neural network, the safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer and a bottleneck layer, and the multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device for real-time data collection and reasoning, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized feature are transmitted to the cloud for review; Optionally, this embodiment builds an edge safety detection model based on a lightweight convolutional neural network for use in construction site safety monitoring scenarios. This safety detection model mainly targets three types of detection tasks: construction workers wearing safety helmets, wearing reflective clothing, and dangerous behaviors. In the specific implementation process, a feature extraction network structure combining a deep convolutional layer and a bottleneck layer is first designed, in which the deep convolutional layer uses a deep separable convolution operation to extract visual features of different scales in a layer-by-layer progressive manner, including low-layer edge texture features, middle-layer shape contour features, and high-level semantic features.

[0025] This embodiment introduces a bottleneck layer structure after the deep convolution layer, which implements channel dimension reduction and dimension increase operations through 1×1 convolution, effectively reducing the amount of model calculation while maintaining feature expression capabilities. The extracted multi-scale features are transformed nonlinearly using the rectified linear unit (ReLU) activation function to eliminate negative responses in the features and enhance the expression of positive features, thereby obtaining an optimized feature map.

[0026] In the edge device deployment phase, this embodiment deploys the optimized lightweight model on an edge computing device with a neural network processor. The real-time video stream data of the construction site is collected through the device's built-in image sensor, and target detection reasoning is performed on each frame of the image. A feature pyramid network is constructed based on the optimized features, and a multi-scale detection strategy is used to accurately locate targets of different sizes, and the detection box position and the corresponding category confidence score are output.

[0027] This embodiment sets up a dual-threshold judgment mechanism to balance detection efficiency and accuracy. When the confidence score is higher than the first threshold, it indicates that the detection result has a high degree of credibility, directly triggering an on-site alarm prompt. When the confidence score is between the first threshold and the second threshold, in order to avoid false alarms, the detection results and the corresponding optimization features are transmitted to the cloud through the network for further review and verification.

[0028] In practical applications, this embodiment can adjust the model structure and parameter configuration according to specific scenarios. For example, in small construction sites, a lighter backbone network can be used; in large construction sites, a deeper feature extraction network can be used. For different types of safety violations, such as high-altitude operations and lifting operations, the detection capability of the model can be expanded by adding corresponding training data.

[0029] This embodiment solves the problems of poor real-time performance and high resource consumption of traditional safety detection systems through lightweight network design and edge computing deployment strategy. The introduction of the dual threshold mechanism effectively balances detection efficiency and accuracy, and improves the practicality of the system in complex construction site environments. The optimized feature extraction method and detection strategy significantly improve the model's detection performance for targets of different scales, providing reliable technical support for construction site safety management.

[0030] Step S102: deploying a deep neural network model as a review system in the cloud, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, and calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with a time series correlation analysis, and updates the alarm status based on the review result and feeds the review result back to the edge; Optionally, this embodiment deploys a deep model using a residual neural network architecture in the cloud as a review system for fine-grained analysis of the data to be reviewed uploaded by the edge. The review system uses a multi-layer residual module stacking method to construct a feature extraction network. Each residual module contains a series structure of multiple convolutional layers, and uses jump connections to effectively alleviate the gradient vanishing problem caused by network deepening.

[0031] This embodiment introduces an attention mechanism and a multi-branch structure based on the residual network to enhance the expressiveness of features. The attention mechanism includes two branches: channel attention and spatial attention. The channel attention branch learns the importance weights between feature channels through global pooling and fully connected layers; the spatial attention branch learns the attention distribution of spatial positions through convolution operations. The multi-branch structure processes features in parallel at different scales to capture multi-scale information of the target.

[0032] For the detection results uploaded by the edge, this embodiment first performs deep semantic feature extraction on the optimized features. The extracted features are enhanced by the attention mechanism and multi-branch structure and used to perform refined analysis, including accurate positioning of the target bounding box and reliability verification of the category label. For example, in a construction site scenario, the review system can accurately identify the helmet wearing judgment error caused by perspective changes or partial occlusion.

[0033] This embodiment innovatively introduces a feature sensitivity analysis mechanism, which generates a feature sensitivity map by calculating the activation response of the enhanced features at each layer of the edge model. This map intuitively shows the degree of attention the model pays to different areas of the input image, which helps to understand the decision basis of the model. Based on the feature sensitivity map, the spatial attention weights of the key areas are further extracted to form a regional weight map.

[0034] In order to improve the time series perception ability of the review system, this embodiment combines the regional weight map with the time series information of the historical frames for analysis. The regional weight changes of multiple consecutive frames are modeled through a recursive neural network to generate a sample importance score that reflects the importance of the current sample to the model optimization. This time series analysis mechanism is particularly suitable for identifying dynamic dangerous behaviors of construction workers, such as high-altitude operations and illegal operations.

[0035] The review results of this embodiment are fed back to the edge end in real time through the message queue, and the alarm status is updated accordingly. In practical applications, the review system can effectively reduce the false alarm rate and improve the reliability of the detection results. For example, when the edge end detects suspicious security violations, the review system can accurately determine whether an alarm needs to be triggered through multi-dimensional analysis.

[0036] By deploying a cloud-based review system, this embodiment solves the problem of insufficient detection accuracy of edge models in complex scenarios. Deep feature extraction based on residual networks, combined with attention mechanisms and timing analysis, significantly improves the system's ability to identify various safety hazards. Feature sensitivity analysis and sample importance assessment mechanisms provide reliable data support for the continuous optimization of the model, enabling the system to continuously adapt to the dynamic changes in the construction site environment.

[0037] Step S103: construct a double cache structure to maintain a training sample pool, the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map form training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, a small batch of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0038] Optionally, this embodiment constructs a dual-buffer structure training sample management mechanism for efficiently managing and utilizing training samples generated during construction site safety inspection. The structure includes an online cache area and an offline storage area, which are used to process sample data of different priorities, respectively, to achieve hierarchical storage and training optimization of samples.

[0039] This embodiment integrates the enhanced features, review results, and regional weight maps obtained from the cloud review system into a complete training sample. Each training sample is accompanied by a sample importance score, which comprehensively considers multiple dimensions such as feature expression ability, detection difficulty, and time series correlation. Based on the sample importance score, all training samples are prioritized to ensure that more valuable samples can be processed in a timely manner.

[0040] In terms of sample storage strategy, this embodiment stores high-value samples with high ranking in the online cache area. These samples usually contain difficult-to-classify cases or newly emerging safety hazard patterns. For example, the recognition results of helmets under low light conditions, the detection results of reflective clothing under occlusion, etc. These samples will be used for real-time model updates to ensure that the model can quickly adapt to new scene changes.

[0041] For samples ranked low, this embodiment stores them in an offline storage area for regular batch training. Such samples usually appear as test results in conventional scenarios, and the basic performance of the model can be maintained through regular training. In each round of model update, small batches of samples are randomly extracted from the online cache area to construct a training set, and the knowledge of the cloud-based review model is migrated to the edge model through knowledge distillation.

[0042] This embodiment uses a contrastive learning strategy to enhance the feature discrimination ability of the model. During the training process, positive sample pairs in the same scene (such as images of correctly wearing a helmet) are used as positive examples, and sample pairs in different scenes or different categories are used as negative examples. The model parameters are optimized by minimizing the positive example feature distance and maximizing the negative example feature distance.

[0043] Based on the results of each round of training, this embodiment dynamically adjusts the learning rate and confidence threshold. When the model performance is significantly improved, the learning rate is reduced to achieve more refined parameter adjustment; when the detection accuracy is improved, the confidence threshold is appropriately increased to reduce the amount of data reviewed in the cloud. This adaptive optimization mechanism ensures that the model can maintain stable detection performance under different working conditions.

[0044] The dual-buffer training mechanism of this embodiment solves the problems of low sample utilization efficiency and fixed update strategy in traditional model update methods. Through hierarchical storage and priority training of samples, timely response to important samples is guaranteed while maintaining the stability of the model in conventional scenarios. The combination of knowledge distillation and contrastive learning improves the feature expression ability of the model, while the dynamic parameter adjustment mechanism ensures the continuous optimization ability of the system, providing reliable technical guarantee for safety monitoring of construction sites.

[0045] From the above description, it can be seen that the security monitoring method based on the hybrid of large models and neural network algorithms provided in the embodiment of the present application can perform real-time detection by deploying lightweight convolutional neural networks at the edge, and screen samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploy deep neural network models in the cloud to perform refined analysis, use residual structures and attention mechanisms to extract deep semantic features, and make review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system; the present application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0046] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 2 , and can also include the following: Step S201: Divide the lightweight convolutional neural network into a feature extraction part and a feature fusion part, wherein the feature extraction part uses 5 layers of depth-separable convolutional layers for feature extraction, the convolution kernel size of each layer of the depth-separable convolutional layer is 3×3, the number of channels is 32, 64, 128, 256, 512, and the step size is 2, and a batch normalization layer is connected after each layer of the depth-separable convolutional layer to standardize the feature data to obtain the multi-scale features; Step S202: three bottleneck layer modules are set in the feature fusion part, each of the bottleneck layer modules includes a series structure of a 1×1 convolution layer, a 3×3 deep convolution layer and a 1×1 convolution layer, the number of output channels of the bottleneck layer modules are 256, 128, and 64 respectively, the multi-scale features are input into the bottleneck layer module, and the output of the bottleneck layer module is nonlinearly transformed by using a rectified linear unit activation function to obtain the optimized features.

[0047] Optionally, this embodiment adopts a lightweight convolutional neural network architecture to achieve efficient target detection through two main parts: feature extraction and feature fusion. In the design of the network structure, special consideration is given to the real-time requirements in complex scenes of construction sites, which significantly reduces the computational overhead while ensuring detection accuracy.

[0048] The feature extraction part of this embodiment adopts a cascade structure of 5 layers of depth-separable convolutional layers. Each convolutional layer uses a convolution kernel of size 3×3. This design achieves a good balance between the size of the receptive field and the computational complexity. The number of channels increases in the order of 32, 64, 128, 256, and 512, so that the network can learn visual features from low-level to high-level layer by layer. For example, the first layer mainly extracts basic features such as edges and textures, while the subsequent layers can recognize the combined features of targets such as helmets and reflective clothing.

[0049] The innovation of the depthwise separable convolution is that it decomposes the standard convolution into two steps: channel-by-channel convolution and point-by-point convolution. This design can effectively reduce the number of parameters and computations when processing monitoring images of construction sites. The step size is set to 2 to ensure that the size of the feature map is halved layer by layer, which not only expands the receptive field but also reduces memory usage. The batch normalization layer connected after each convolution layer improves the stability of the features through standardization processing, effectively coping with the influence of external factors such as lighting changes and weather conditions.

[0050] In the feature fusion part, this embodiment designs three bottleneck layer modules, each of which adopts a series structure of 1×1 convolution layer, 3×3 deep convolution layer and 1×1 convolution layer. This design draws on the idea of ​​residual network, but greatly reduces the computational complexity by using a combination of deep convolution and 1×1 convolution. The number of output channels decreases from 256 to 64, achieving a gradual reduction in feature dimension, which is conducive to extracting more compact feature representation.

[0051] The first 1×1 convolutional layer of the bottleneck layer module is used for dimensionality reduction, reducing the amount of computation for subsequent deep convolutions. The 3×3 deep convolutional layer is responsible for extracting spatial features, while the second 1×1 convolutional layer is used for feature dimensionality increase and information integration. This design significantly reduces the number of model parameters while maintaining the feature expression capability. The nonlinear transformation of the modified linear unit activation function further enhances the feature discrimination capability.

[0052] The design of this embodiment solves the problem that traditional deep networks are difficult to deploy on edge devices. Through the combination of deep separable convolution and bottleneck layer structure, the computational complexity of the model is significantly reduced, enabling real-time detection on edge devices on construction sites. The multi-level feature extraction and fusion mechanism ensures the model's ability to detect targets of different scales, especially in complex construction site environments, and can accurately identify safety protection equipment at different distances and angles.

[0053] The network structure has shown good detection performance and operation efficiency in practical applications. It can quickly and accurately identify whether workers are wearing protective equipment such as safety helmets and reflective clothing correctly, providing reliable technical support for construction site safety management. The introduction of batch normalization layers improves the model's adaptability under different environmental conditions and ensures the stability of the detection results.

[0054] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 3 , and can also include the following: Step S301: quantize and compress the security detection model, convert the model weight from a floating point number to an 8-bit integer, divide the security detection model into 32 computing units, deploy the computing units in the neural network processor of the edge device in a parallel computing manner, collect video data with a resolution of 1920×1080 based on the image sensor of the edge device, and parse the video data into image frames at a frequency of 30 frames per second to obtain the real-time collected data; Step S302: input the real-time collected data into the security detection model, construct a three-level feature pyramid network through the optimized features, generate candidate areas by using anchor box prediction, perform non-maximum suppression on the candidate areas to remove overlapping boxes, calculate the category probability distribution of the candidate areas to obtain the detection results, and use the softmax function to normalize the category probability distribution to obtain the confidence score.

[0055] Optionally, this embodiment first performs quantization compression optimization on the security detection model, and significantly reduces the model storage space requirement and computational complexity by converting the model weights from floating point numbers to 8-bit integers. This quantization compression process maintains the model performance while making it more suitable for deployment and operation on edge devices with limited computing resources.

[0056] This embodiment divides the compressed model into 32 computing units, making full use of the parallel computing capabilities of the edge device neural network processor. Each computing unit is responsible for a specific network layer computing task, and the model reasoning speed is improved through parallel processing. In actual applications on construction sites, this parallel computing architecture can support the real-time video analysis needs of multiple monitoring points.

[0057] Based on the image sensor of the edge device, this embodiment collects 1920×1080 high-definition resolution video data and analyzes it at a frequency of 30 frames per second to ensure real-time monitoring of the construction site. High-resolution video data helps capture detailed information, such as the wearing status of safety helmets, the compliance of reflective clothing, and other safety protection details.

[0058] In the target detection stage, this embodiment constructs a three-level feature pyramid network based on optimized features. This multi-scale feature representation method can effectively handle detection targets of different distances and sizes in the construction site. For example, clear images of nearby workers and blurred images of distant workers can be accurately detected.

[0059] This embodiment uses an anchor box prediction mechanism to generate candidate regions. The pre-set anchor box size and proportion are optimized for common targets on construction sites (such as safety helmets, reflective clothing, etc.). Through non-maximum suppression processing, overlapping detection frames are effectively removed to improve the accuracy of detection results. This is especially important in densely populated areas, and can accurately distinguish the safety equipment status of adjacent workers.

[0060] For each candidate area, this embodiment calculates its category probability distribution, including multiple categories such as correctly wearing a helmet, not wearing a helmet, and wearing a reflective vest in a standardized manner. The probability distribution is normalized by the softmax function to obtain a reliable confidence score. This scoring mechanism provides an important basis for subsequent alarm judgment.

[0061] The innovative design of this embodiment solves the real-time and accuracy problems faced by traditional security detection systems when deployed at the edge. The combination of quantization compression and parallel computing significantly improves the operating efficiency of the model on edge devices, enabling the system to detect and respond to security risks in a timely manner. The multi-scale feature representation of the feature pyramid network enhances the model's adaptability to different scenarios, while the carefully designed post-processing mechanism ensures the reliability of the detection results.

[0062] In practical applications, this embodiment can achieve real-time detection of safety protection equipment and timely discovery of violations in complex construction site environments. The system runs reliably on various edge devices, providing strong technical support for safety management at construction sites and significantly improving the efficiency and accuracy of safety supervision.

[0063] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 4 , and can also include the following: Step S401: construct a 50-layer residual neural network, the residual neural network includes four residual module groups, each of the residual module groups is composed of multiple residual units, the residual unit adopts a series structure of 1×1 convolution layer, 3×3 convolution layer, and 1×1 convolution layer, the number of output channels of the residual module group is 256, 512, 1024, and 2048 respectively, the optimized features are input into the residual neural network for feature transformation to obtain the deep semantic features; Step S402: construct a channel attention branch and a spatial attention branch, the channel attention branch uses a global average pooling layer and a fully connected layer to generate a channel weight, and the spatial attention branch uses a convolutional layer and a maximum pooling layer to generate a spatial weight, and the channel weight and the spatial weight are respectively element-wise multiplied with the deep semantic features to obtain the enhanced features, and the enhanced features are used to correct the target bounding box and category label of the detection result to obtain the review result.

[0064] Optionally, this embodiment constructs a deep feature extraction architecture based on a 50-layer residual neural network for deep semantic analysis of the optimized features uploaded by the edge. The network includes four residual module groups, each of which is composed of multiple residual units connected in series, and hierarchical feature extraction is achieved by deepening layer by layer.

[0065] Each residual unit adopts the design structure of "1×1 convolution-3×3 convolution-1×1 convolution", in which the first 1×1 convolution is used for dimensionality reduction to reduce the computational complexity of the subsequent 3×3 convolution; the 3×3 convolution is responsible for extracting the main features; and the last 1×1 convolution is used for feature dimensionality increase. This bottleneck structure design demonstrates excellent feature extraction capabilities in construction site safety monitoring scenarios and can effectively capture the detailed features of safety protection equipment.

[0066] In this embodiment, the number of output channels of the four residual module groups is 256, 512, 1024, and 2048, respectively. As the network depth increases, the feature dimension gradually increases. This progressive design enables the network to transition from shallow basic visual features (such as edges and textures) to deep semantic features (such as the integrity of the helmet, the wearing position, etc.), providing a reliable foundation for subsequent refined analysis.

[0067] To enhance the expressiveness of features, this embodiment innovatively introduces a dual-branch attention mechanism. The channel attention branch captures the global information of the feature map through global average pooling, and then learns the correlation between channels through the fully connected layer to generate channel weights. This design can highlight important feature channels, such as feature maps related to safety equipment.

[0068] The spatial attention branch uses a combination of convolutional layers and maximum pooling layers to generate attention weights in the spatial dimension. This enables the network to focus on key areas in the image, such as the worker's head, upper body, and other main wearing locations of safety protection equipment. For example, when detecting the wearing status of a hard hat, the spatial attention mechanism automatically focuses on the worker's head area to improve detection accuracy.

[0069] This embodiment multiplies the generated channel weights and spatial weights by the deep semantic features element by element to achieve adaptive enhancement of features. This dual enhancement mechanism can effectively improve the model's ability to understand complex scenes, especially in challenging scenes such as poor lighting conditions and crowded people.

[0070] After obtaining the enhanced features, this embodiment uses these features to perform fine correction on the detection results uploaded by the edge. The correction process includes fine-tuning the position of the target bounding box and verifying the reliability of the category label to ensure the accuracy of the detection results. For example, when the edge model has uncertainty in judging the wearing status of a helmet under partial occlusion, the cloud-based review system can provide more reliable judgment results through enhanced features.

[0071] This embodiment effectively solves the problem of insufficient accuracy of traditional detection systems in complex construction site environments. The combination of deep residual networks and attention mechanisms significantly improves the system's ability to identify various types of safety protection equipment. Especially in scenarios where the light on the construction site changes greatly and there are many occlusions, the solution shows good robustness and provides reliable technical support for construction site safety management.

[0072] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 5 , and can also include the following: Step S501: input the enhanced feature into the edge security detection model, record the feature map of each convolutional layer in the edge model, calculate the gradient value of the enhanced feature to the feature map of each layer by back propagation, perform global average pooling and L2 normalization on the gradient value, and perform weighted summation of the normalized gradient value and the feature map of the corresponding layer to obtain the feature sensitivity map; Step S502: perform adaptive threshold segmentation on the feature sensitivity map to extract high response areas, calculate the spatial position and area ratio of the high response areas in the original image to obtain the regional weight map, construct a long short-term memory network to perform time series analysis on the regional weight map in 10 consecutive frames of video, perform weighted fusion of the time series analysis results and the spatial statistical features of the regional weight map to obtain the sample importance score, perform credibility assessment on the review results based on the sample importance score, feed back the review results to the edge end through a message queue and update the alarm status.

[0073] Optionally, this embodiment designs a deep learning interpretability solution based on feature sensitivity analysis, which implements reliability assessment of safety detection results by carefully analyzing the feature response of the edge model. This solution is of great significance in construction site safety monitoring scenarios and can effectively identify and verify the credibility of detection results.

[0074] This embodiment first inputs the enhanced features into the edge safety detection model and records the feature maps of each convolutional layer of the model. The gradient values ​​of the enhanced features to the feature maps of each layer are calculated through the back propagation mechanism. These gradient values ​​reflect the contribution of features at different levels to the final detection results. For example, when identifying the wearing status of a helmet, the feature map related to the head area usually obtains a higher gradient value.

[0075] The obtained gradient values ​​are subjected to global average pooling to obtain the importance evaluation of each feature channel. The subsequent L2 normalization process ensures the comparability of features at different levels. The feature sensitivity map is generated by weighted summing the normalized gradient values ​​with the feature map of the corresponding layer, which intuitively shows the key areas of the model's attention.

[0076] This embodiment uses an adaptive threshold segmentation method to extract high-response areas from the feature sensitivity map. These areas usually correspond to key parts of safety equipment, such as the outline of a helmet, the logo of a reflective vest, etc. By calculating the spatial position and area proportion of these high-response areas in the original image, a regional weight map is generated to quantify the importance of different areas.

[0077] In order to make full use of the timing information in the video sequence, this embodiment innovatively introduces a long short-term memory network to perform timing analysis on the regional weight map in 10 consecutive frames of video. This design can effectively capture the dynamic changes in the use status of workers' safety equipment and improve the stability of the detection results. For example, when a worker is adjusting the position of a helmet, timing analysis can avoid misjudgment due to transient status.

[0078] This embodiment performs a weighted fusion of the time series analysis results and the spatial statistical features of the regional weight map to obtain a sample importance score. This score takes into account information from both spatial and temporal dimensions and can more comprehensively evaluate the reliability of the test results. For example, if the key area of ​​a test result continues to show a stable high response, the result is more credible.

[0079] Based on the sample importance score, this embodiment evaluates the credibility of the review results, and feeds back the evaluation results to the edge through the message queue to update the alarm status in real time. This instant feedback mechanism ensures the timeliness and accuracy of security supervision.

[0080] This embodiment effectively solves the problem of insufficient interpretability of traditional safety detection systems by combining feature sensitivity analysis and time series modeling. This solution can not only accurately identify safety hazards, but also provide reliability assessment of detection results, providing strong support for construction site safety management decisions. In practical applications, this explainable detection mechanism significantly improves the credibility of the system, reduces the occurrence of false positives and false negatives, and makes safety supervision more accurate and efficient.

[0081] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 6 , and can also include the following: Step S601: Establish a sample database based on key-value pair storage in the cloud server, set the online cache area as a high-speed memory database, the storage capacity of the online cache area is 1000 training samples, set the offline storage area as a solid-state hard disk database, the storage capacity of the offline storage area is 10000 training samples, assign a unique identifier to each of the training samples, serialize the enhanced features, the review results and the regional weight map, and store them in association with the identifier; Step S602: normalize the sample importance score to the range of 0 to 1, set the sample importance threshold to 0.8, arrange the training samples in descending order according to the sample importance score, write the training samples with scores greater than the sample importance threshold into the online cache area, delete the sample with the lowest score when the number of samples in the online cache area exceeds the storage capacity, write the training samples with scores less than the sample importance threshold into the offline storage area, and perform batch training updates on the samples in the offline storage area every 24 hours.

[0082] Optionally, this embodiment constructs an efficient two-layer storage architecture to achieve dynamic management of training samples and model updates through cloud servers. The architecture uses key-value pair storage to establish a sample database, which includes two levels: a high-speed memory database and a solid-state drive database, which are used to store high-value and ordinary-value training samples respectively.

[0083] In this embodiment, the online cache area is configured as a high-speed memory database and the storage capacity is set to 1,000 training samples. This configuration makes full use of the high-speed read and write characteristics of the memory to ensure fast access to important samples. The online cache area is mainly used to store high-value samples such as those with typical construction site safety protection cases, such as helmet wearing recognition under special lighting conditions and reflective clothing detection under complex backgrounds.

[0084] The offline storage area is configured as a solid-state drive database with a large storage space for 10,000 training samples. This design is suitable for storing training samples in common scenarios, while ensuring storage cost-effectiveness and providing sufficient training data for continuous optimization of the model.

[0085] This embodiment assigns a unique identifier to each training sample, and stores the enhanced features, review results, and regional weight maps in association with the identifier after serialization. This organization method ensures the integrity and traceability of the data, and facilitates subsequent sample retrieval and model training. For example, when it is necessary to analyze the detection effect of a certain type of safety hazard, the relevant samples can be quickly located.

[0086] The storage management of samples adopts a dynamic scheduling mechanism based on importance scores. This embodiment normalizes the sample importance scores to a range of 0 to 1, and sets 0.8 as the importance threshold. Samples above the threshold are considered to have higher training value, such as the first type of security risks, more difficult to identify violations, etc. These samples will be written to the online cache area first to ensure that the model can quickly learn these key cases.

[0087] When the number of samples in the online cache reaches the storage limit, this embodiment maintains efficient use of storage space by deleting the samples with the lowest scores. This dynamic update mechanism ensures that the online cache always keeps the most valuable training samples. Samples below the importance threshold are written to the offline storage area. Although these samples are less important, they are still helpful in improving the generalization ability of the model.

[0088] This embodiment sets a batch training update for the samples in the offline storage area every 24 hours. This periodic update strategy balances the demand for computing resource consumption and model performance improvement. Through regular batch training, the model can gradually adapt to changes in the construction site safety monitoring scene and improve the accuracy and robustness of detection.

[0089] This embodiment effectively solves the difficulties of traditional safety detection systems in sample management and model updating through hierarchical storage and dynamic management. This design not only optimizes the efficiency of storage resource utilization, but also ensures that the model can continue to learn from high-quality samples, improving the accuracy and real-time performance of construction site safety monitoring. In practical applications, this solution significantly improves the reliability and adaptability of the safety detection system, providing more intelligent and efficient technical support for construction site safety management.

[0090] In one embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm of the present application, see Figure 7 , and can also include the following: Step S701: A stratified sampling strategy is adopted to obtain training samples from the online cache area, each small batch contains 32 training samples, and the stratified sampling is balanced based on the sample distribution of three types of tasks: helmet wearing, reflective clothing wearing, and personnel behavior. The enhanced features of the training samples are input into the teacher model to obtain soft labels, and the soft labels are paired with the review results to construct the training set; Step S702: input the training set into the student model to obtain a prediction result, calculate the cross entropy between the prediction result and the soft label to obtain the distillation loss, construct a feature contrast loss function to measure the similarity between the output features of the student model and the teacher model, take the weighted sum of the distillation loss and the feature contrast loss as the total loss function, use the stochastic gradient descent method to optimize the total loss function to update the student model parameters, adaptively adjust the learning rate based on the rate of change of the total loss function, and dynamically update the confidence threshold according to the accuracy on the validation set.

[0091] Optionally, this embodiment designs a model optimization solution based on knowledge distillation to improve the performance of the edge safety detection model through stratified sampling and multi-task balanced learning. This solution pays special attention to three core tasks in construction site safety monitoring: helmet wearing detection, reflective clothing wearing detection and personnel behavior analysis.

[0092] This embodiment uses a stratified sampling strategy to obtain training samples from the online cache, and each small batch contains 32 samples. This sampling method ensures a balanced distribution of samples for the three core tasks and avoids task bias during model training. For example, when sampling, multiple scenarios such as not wearing a helmet, wearing reflective clothing in accordance with regulations, and illegal operations will be considered at the same time, so that the model can fully learn the characteristics of various safety hazards.

[0093] By inputting the enhanced features of the training samples into the teacher model, soft label information is obtained. These soft labels contain the probability distribution predictions of the teacher model for various safety hazards, which can provide more fine-grained learning objectives for the student model. This embodiment pairs the soft labels with the cloud review results to construct a high-quality training set.

[0094] In the model training phase, this embodiment inputs the training set into the lightweight student model and obtains the distillation loss by calculating the cross entropy between the prediction results and the soft labels. This loss function design enables the student model to not only learn the correct prediction results, but also imitate the decision-making process of the teacher model.

[0095] This embodiment innovatively introduces a feature contrast loss function to measure the similarity between the student model and the teacher model at the feature expression level. This design ensures that the student model can learn the key features extracted by the teacher model. For example, when identifying the wearing status of a helmet, the student model will learn to focus on the key feature points of the head area.

[0096] The total loss function adopts a weighted combination of distillation loss and feature contrast loss, and the weights can be dynamically adjusted according to actual needs. This embodiment uses the stochastic gradient descent method to optimize the total loss function and iteratively updates the parameters of the student model. The adaptive adjustment mechanism of the learning rate ensures the stability and convergence efficiency of the model training.

[0097] This embodiment dynamically updates the confidence threshold based on the accuracy on the validation set. This adaptive mechanism enables the model to adjust the decision criteria based on the actual detection effect. For example, when a large change in lighting conditions is detected, the system will adjust the confidence threshold accordingly to maintain the reliability of the detection.

[0098] This embodiment effectively solves the problem of improving the performance of edge models under limited computing resources by combining knowledge distillation and feature contrast learning. This solution not only achieves the lightweight of the model, but also maintains a high detection accuracy. In practical applications, the optimized edge model can quickly respond to various safety hazards and provide timely and reliable early warning information for construction site safety management.

[0099] Through the design of stratified sampling and multi-task balanced learning, this embodiment significantly improves the model's ability to identify various safety hazards. Especially in complex construction site environments, the solution shows good generalization performance and stability, providing more reliable technical support for construction site safety supervision.

[0100] In order to improve the detection accuracy and real-time response capability of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks, the present application provides an embodiment of a security monitoring device based on a hybrid of a large model and a neural network algorithm for implementing all or part of the security monitoring method based on a hybrid of a large model and a neural network algorithm, see Figure 8 The safety monitoring device based on the hybrid of large model and neural network algorithm specifically includes the following contents: A confidence scoring module 10 is used to build an edge safety detection model based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolution layer and a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device to perform real-time data collection and reasoning. Target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized features are transmitted to the cloud for review. A review alarm module 20 is used to deploy a deep neural network model in the cloud as a review system, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, performs a refined analysis on the detection results based on the enhanced features to obtain a review result, calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, combines the regional weight map with time series correlation analysis to generate a sample importance score, updates the alarm state based on the review result and feeds the review result back to the edge; The dynamic update module 30 is used to construct a dual cache structure to maintain a training sample pool, wherein the training sample pool includes an online cache area and an offline storage area, wherein the enhanced features, the review results and the regional weight map form a training sample, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for periodic batch training, and small batches of samples are extracted from the online cache area in each round of update to construct a training set, and the training set is used for knowledge distillation to obtain a distillation loss, and model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0101] From the above description, it can be seen that the security monitoring device based on the hybrid of large models and neural network algorithms provided by the embodiment of the present application can perform real-time detection by deploying lightweight convolutional neural networks at the edge, and screen samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploy deep neural network models in the cloud to perform refined analysis, extract deep semantic features using residual structures and attention mechanisms, and make review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system; the present application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0102] From the hardware level, in order to improve the detection accuracy and real-time response capability of the system by building an intelligent framework of edge-cloud collaboration and combining the advantages of large models and lightweight neural networks, the present application provides an embodiment of an electronic device for implementing all or part of the security monitoring method based on a hybrid of large models and neural network algorithms, and the electronic device specifically includes the following contents: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the safety monitoring device based on the hybrid of large model and neural network algorithm and related equipment such as core business system, user terminal and related database; the logic controller can be a desktop computer, tablet computer and mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the safety monitoring method based on the hybrid of large model and neural network algorithm and the embodiment of the safety monitoring device based on the hybrid of large model and neural network algorithm in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0103] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0104] In practical applications, part of the security monitoring method based on the hybrid of the large model and the neural network algorithm can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0105] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0106] Fig. 9FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0107] In one embodiment, the safety monitoring method function based on the hybrid of the large model and the neural network algorithm can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control: Step S101: construct an edge safety detection model based on a lightweight convolutional neural network, the safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer and a bottleneck layer, and the multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device for real-time data collection and reasoning, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized feature are transmitted to the cloud for review; Step S102: deploying a deep neural network model as a review system in the cloud, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, and calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with a time series correlation analysis, and updates the alarm status based on the review result and feeds the review result back to the edge; Step S103: construct a double cache structure to maintain a training sample pool, the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map form training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, a small batch of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0108] From the above description, it can be seen that the electronic device provided by the embodiment of the present application performs real-time detection by deploying a lightweight convolutional neural network at the edge, and screens samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploys a deep neural network model in the cloud to perform refined analysis, extracts deep semantic features using residual structures and attention mechanisms, and makes review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system; the present application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0109] In another embodiment, the safety monitoring device based on the mixture of large model and neural network algorithm can be configured separately from the central processing unit 9100. For example, the safety monitoring device based on the mixture of large model and neural network algorithm can be configured as a chip connected to the central processing unit 9100, and the function of the safety monitoring method based on the mixture of large model and neural network algorithm can be realized through the control of the central processing unit.

[0110] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0111] like Fig. 9As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0112] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0113] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0114] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0115] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0116] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0117] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0118] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the security monitoring method based on a combination of a large model and a neural network algorithm in the above-mentioned embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the security monitoring method based on a combination of a large model and a neural network algorithm in the above-mentioned embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: construct an edge safety detection model based on a lightweight convolutional neural network, the safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer and a bottleneck layer, and the multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device for real-time data collection and reasoning, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized feature are transmitted to the cloud for review; Step S102: deploying a deep neural network model as a review system in the cloud, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, and calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with a time series correlation analysis, and updates the alarm status based on the review result and feeds the review result back to the edge; Step S103: construct a double cache structure to maintain a training sample pool, the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map form training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, a small batch of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0119] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application performs real-time detection by deploying a lightweight convolutional neural network at the edge, and screens samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploys a deep neural network model in the cloud to perform refined analysis, extracts deep semantic features using residual structures and attention mechanisms, and makes review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system; the present application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0120] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the safety monitoring method based on the hybrid of a large model and a neural network algorithm in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the safety monitoring method based on the hybrid of a large model and a neural network algorithm are implemented. For example, the computer program / instruction implements the following steps: Step S101: construct an edge safety detection model based on a lightweight convolutional neural network, the safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer and a bottleneck layer, and the multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device for real-time data collection and reasoning, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized feature are transmitted to the cloud for review; Step S102: deploying a deep neural network model as a review system in the cloud, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, and calculates the activation response of the enhanced features at each layer of the edge model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with a time series correlation analysis, and updates the alarm status based on the review result and feeds the review result back to the edge; Step S103: construct a double cache structure to maintain a training sample pool, the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map form training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, a small batch of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and the confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

[0121] From the above description, it can be seen that the computer program product provided by the embodiment of the present application performs real-time detection by deploying a lightweight convolutional neural network at the edge, and screens samples to be reviewed through multi-scale feature extraction and confidence scoring mechanisms; deploys a deep neural network model in the cloud to perform refined analysis, extracts deep semantic features using residual structures and attention mechanisms, and makes review decisions based on feature sensitivity and regional weights. At the same time, a double cache structure is constructed to maintain the training sample pool, and a differentiated update strategy is implemented according to the importance of the samples. The model performance is continuously optimized through knowledge distillation and comparative learning, which effectively balances the edge computing resource constraints and detection accuracy requirements, and realizes the intelligent upgrade of the security monitoring system; the present application can improve the detection accuracy and real-time response of the system by building an intelligent framework for edge-cloud collaboration and combining the advantages of large models and lightweight neural networks.

[0122] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A safety monitoring method based on a hybrid of a large model and a neural network algorithm, characterized in that: The method comprises: An edge safety detection model is constructed based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted through a combination of a deep convolutional layer and a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device to perform real-time data collection and reasoning to obtain real-time collected data. Target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized features are transmitted to the cloud for review. A deep neural network model is deployed in the cloud as a review system, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, and combines the attention mechanism and the multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain a review result, and calculates the activation response of the enhanced features at each layer of the edge security detection model to obtain a feature sensitivity map, and extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, and generates a sample importance score by combining the regional weight map with the time series correlation analysis, and updates the alarm status based on the review result and feeds the review result back to the edge; A double cache structure is constructed to maintain a training sample pool, wherein the training sample pool includes an online cache area and an offline storage area, the enhanced features, the review results and the regional weight map are combined into training samples, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate updating, and the training samples with lower rankings are stored in the offline storage area for periodic batch training, small batches of samples are extracted from the online cache area in each round of updating to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

2. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: The edge safety detection model is constructed based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolutional layer with a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features, including: The lightweight convolutional neural network is divided into a feature extraction part and a feature fusion part. The feature extraction part uses 5 layers of depth-separable convolutional layers for feature extraction. The convolution kernel size of each depth-separable convolutional layer is 3×3, the number of channels is 32, 64, 128, 256, and 512, respectively, and the step size is 2. After each depth-separable convolutional layer, a batch normalization layer is connected to standardize the feature data to obtain the multi-scale features; Three bottleneck layer modules are set in the feature fusion part, each of the bottleneck layer modules includes a series structure of a 1×1 convolution layer, a 3×3 deep convolution layer and a 1×1 convolution layer, and the number of output channels of the bottleneck layer modules is 256, 128, and 64, respectively. The multi-scale features are input into the bottleneck layer module, and the output of the bottleneck layer module is nonlinearly transformed by using a rectified linear unit activation function to obtain the optimized features.

3. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: The security detection model is deployed on the edge device to perform real-time data collection and reasoning to obtain real-time collected data, and target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores, including: The security detection model is quantized and compressed, the model weight is converted from a floating point number to an 8-bit integer, the security detection model is divided into 32 computing units, the computing units are deployed in the neural network processor of the edge device in a parallel computing manner, and video data with a resolution of 1920×1080 is collected based on the image sensor of the edge device, and the video data is parsed into image frames at a frequency of 30 frames per second to obtain the real-time collected data; The real-time collected data is input into the security detection model, a three-level feature pyramid network is constructed through the optimized features, a candidate area is generated by an anchor box prediction method, non-maximum suppression is performed on the candidate area to remove overlapping boxes, the category probability distribution of the candidate area is calculated to obtain the detection result, and the category probability distribution is normalized using a softmax function to obtain the confidence score.

4. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: The deep neural network model is deployed in the cloud as a review system. The review system uses a residual neural network architecture to extract deep semantic features from the optimized features, combines the attention mechanism and the multi-branch structure to enhance the deep semantic features to obtain enhanced features, and performs a refined analysis on the detection results based on the enhanced features to obtain review results, including: Constructing a 50-layer residual neural network, wherein the residual neural network includes four residual module groups, each of which is composed of a plurality of residual units, wherein the residual unit adopts a serial structure of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer, and the number of output channels of the residual module group is 256, 512, 1024, and 2048, respectively, and inputting the optimized features into the residual neural network for feature transformation to obtain the deep semantic features; A channel attention branch and a spatial attention branch are constructed, wherein the channel attention branch uses a global average pooling layer and a fully connected layer to generate a channel weight, and the spatial attention branch uses a convolutional layer and a maximum pooling layer to generate a spatial weight. The channel weight and the spatial weight are respectively element-wise multiplied with the deep semantic features to obtain the enhanced features, and the enhanced features are used to correct the target bounding box and the category label of the detection result to obtain the review result.

5. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: The step of calculating the activation response of the enhanced feature at each layer of the edge security detection model to obtain a feature sensitivity map, extracting the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, combining the regional weight map with time series correlation analysis to generate a sample importance score, updating the alarm status based on the review result and feeding back the review result to the edge end includes: Input the enhanced feature into the edge security detection model, record the feature map of each convolutional layer in the edge model, calculate the gradient value of the enhanced feature to the feature map of each layer by back propagation, perform global average pooling and L2 normalization on the gradient value, and perform weighted summation of the normalized gradient value and the feature map of the corresponding layer to obtain the feature sensitivity map; Perform adaptive threshold segmentation on the feature sensitivity map to extract high response areas, calculate the spatial position and area ratio of the high response area in the original image to obtain the regional weight map, construct a long short-term memory network to perform time series analysis on the regional weight map in 10 consecutive frames of video, perform weighted fusion of the time series analysis results and the spatial statistical features of the regional weight map to obtain the sample importance score, perform credibility assessment on the review results based on the sample importance score, feed back the review results to the edge end through a message queue and update the alarm status.

6. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: The dual buffer structure is constructed to maintain a training sample pool, the training sample pool includes an online buffer area and an offline storage area, the enhanced features, the review results and the regional weight map form a training sample, the training samples are prioritized based on the sample importance scores, the training samples with the highest priority are stored in the online buffer area for immediate update, and the training samples with the lowest priority are stored in the offline storage area for periodic batch training, including: A sample database based on key-value pair storage is established in the cloud server, the online cache area is set as a high-speed memory database, the storage capacity of the online cache area is 1000 training samples, the offline storage area is set as a solid-state hard disk database, the storage capacity of the offline storage area is 10000 training samples, a unique identifier is assigned to each of the training samples, and the enhanced features, the review results and the regional weight map are serialized and stored in association with the identifier; The sample importance score is normalized to the range of 0 to 1, and the sample importance threshold is set to 0.

8. The training samples are arranged in descending order according to the sample importance score, and the training samples with scores greater than the sample importance threshold are written into the online cache area. When the number of samples in the online cache area exceeds the storage capacity, the sample with the lowest score is deleted, and the training samples with scores less than the sample importance threshold are written into the offline storage area. The samples in the offline storage area are batch-trained and updated every 24 hours.

7. The safety monitoring method based on the hybrid of large model and neural network algorithm according to claim 1 is characterized in that: In each round of updating, a small batch of samples is extracted from the online cache area to construct a training set, the training set is used to perform knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and confidence threshold are dynamically adjusted according to the update result to perform system adaptive optimization, including: A stratified sampling strategy is adopted to obtain training samples from the online cache area, each small batch contains 32 training samples, the stratified sampling is balanced based on the sample distribution of three types of tasks: helmet wearing, reflective clothing wearing and personnel behavior, the enhanced features of the training samples are input into the teacher model to obtain soft labels, and the soft labels are paired with the review results to construct the training set; The training set is input into the student model to obtain a prediction result, the cross entropy between the prediction result and the soft label is calculated to obtain the distillation loss, a feature contrast loss function is constructed to measure the similarity between the output features of the student model and the teacher model, a weighted sum of the distillation loss and the feature contrast loss is used as the total loss function, the total loss function is optimized using the stochastic gradient descent method to update the student model parameters, the learning rate is adaptively adjusted based on the rate of change of the total loss function, and the confidence threshold is dynamically updated according to the accuracy on the validation set.

8. A safety monitoring device based on a hybrid of a large model and a neural network algorithm, characterized in that: The device comprises: A confidence scoring module is used to build an edge safety detection model based on a lightweight convolutional neural network. The safety detection model includes three types of detection tasks: helmet wearing, reflective clothing wearing, and personnel behavior. Multi-scale features are extracted by combining a deep convolution layer and a bottleneck layer. The multi-scale features are optimized using a rectified linear unit activation function to obtain optimized features. The safety detection model is deployed on an edge device to perform real-time data collection and inference to obtain real-time collected data. Target detection is performed on the real-time collected data based on the optimized features to obtain detection results and confidence scores. When the confidence score is higher than a first threshold, an alarm is directly triggered. When the confidence score is between the first threshold and the second threshold, the detection result and the optimized features are transmitted to the cloud for review. A review alarm module is used to deploy a deep neural network model in the cloud as a review system, wherein the review system uses a residual neural network architecture to extract deep semantic features from the optimized features, combines the attention mechanism with a multi-branch structure to enhance the deep semantic features to obtain enhanced features, performs a refined analysis on the detection results based on the enhanced features to obtain a review result, calculates the activation response of the enhanced features at each layer of the edge security detection model to obtain a feature sensitivity map, extracts the spatial attention weights of key areas based on the feature sensitivity map to obtain a regional weight map, combines the regional weight map with time series correlation analysis to generate a sample importance score, updates the alarm status based on the review result and feeds the review result back to the edge; A dynamic update module is used to build a double cache structure to maintain a training sample pool, wherein the training sample pool includes an online cache area and an offline storage area, wherein the enhanced features, the review results and the regional weight map form a training sample, the training samples are prioritized based on the sample importance scores, the training samples with higher rankings are stored in the online cache area for immediate update, and the training samples with lower rankings are stored in the offline storage area for regular batch training, small batches of samples are extracted from the online cache area in each round of update to construct a training set, the training set is used for knowledge distillation to obtain a distillation loss, model parameters are updated based on the distillation loss and contrastive learning, and the learning rate and confidence threshold are dynamically adjusted according to the update results to perform system adaptive optimization.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the safety monitoring method based on the hybrid of large model and neural network algorithm as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the safety monitoring method based on the hybrid of large model and neural network algorithm as described in any one of claims 1 to 7 are implemented.

Images