An image processing-based security detection method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of based on image processing's safety detection method, device, electronic equipment and storage medium, specifically related to safety detection technical field, method includes: obtaining the equipment running condition information of each equipment in target factory area and each monitoring image sequence, the equipment information of same condition is grouped, and image sequence is divided according to region;Each group of condition information is input into working state analysis model, and the equipment running state is output;When there is exception, abnormal state information and corresponding image sequence are input into safety detection model, image features and time sequence features are extracted and fused analysis, and safety hazard identification result is output and safety detection is carried out accordingly.The application is used to solve the problem that only using sensor detection, false alarm caused by environmental factors interference in sensor monitoring or abnormal alarm caused by equipment real fault cannot be effectively distinguished, resulting in a large number of false alarms, and real alarm information is submerged.

Description

Technical Field

[0001] This invention relates to the field of security detection technology, and in particular to a security detection method, apparatus, electronic device, and storage medium based on image processing. Background Technology

[0002] Safety monitoring in industrial plants is a core component of ensuring production continuity and personnel safety. Currently, plant equipment safety monitoring primarily relies on sensor-based monitoring. This involves deploying sensors for temperature, vibration, and current on equipment to collect operating parameters and determine whether the equipment is within its normal operating range based on preset thresholds. This reflects the internal operating status of the equipment and identifies potential safety hazards. However, when sensor data is affected by surrounding environmental factors, it is highly susceptible to false alarms or missed detections. Deviations in equipment parameters from thresholds may stem from genuine malfunctions or non-malfunction factors such as environmental interference. Sensors themselves cannot distinguish between these, leading to numerous false alarms. Simultaneously, genuine safety hazards that require attention are buried under a sea of ​​false alarms and overlooked. Summary of the Invention

[0003] This invention proposes a security detection method, device, electronic device, and storage medium based on image processing, which solves the technical problem that existing detection methods, when using only sensors for detection, cannot effectively distinguish between false alarms caused by environmental interference or abnormal alarms caused by actual equipment failures, resulting in a large number of false alarms and the drowning out of real alarm information.

[0004] An embodiment of the present invention provides a security detection method based on image processing, comprising: Acquire equipment operating status information and monitoring image sequences for each piece of equipment in the target plant area; The equipment operating condition information under the same working condition is divided into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment; the monitoring image sequence is divided into the monitoring image sequence corresponding to each region. For each set of equipment, the corresponding equipment operating condition information is input into a pre-trained working state analysis model so that the working state analysis model can analyze and process the equipment operating condition information and output the operating state information of each piece of equipment in the set of equipment. When there is abnormal operating status information, the monitoring area to which the device corresponding to the abnormal operating status information belongs is determined, and the monitoring image sequence corresponding to the monitoring area is obtained; the abnormal operating status information and the corresponding monitoring image sequence are input into a pre-trained safety detection model, so that the safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features; Safety inspections are conducted on the target factory area based on the identified safety hazards.

[0005] Furthermore, the equipment operating condition information under the same working condition is divided into the same data group, including: The operating condition fluctuation coefficient for acquiring operating condition information of each device; Calculate the difference between the operating condition fluctuation coefficients corresponding to the operating condition information of each pair of devices, and compare the difference with a preset difference threshold: If the difference is less than the set difference threshold, it is determined that the operating conditions of the two devices belong to the same operating condition; If the difference is greater than or equal to the set difference threshold, it is determined that the operating conditions of the two devices do not belong to the same operating condition. The operating condition information of all equipment belonging to the same operating condition is grouped into the same data group.

[0006] Furthermore, the step of dividing each monitoring image sequence into monitoring image sequences corresponding to each region includes: Obtain the start and end shooting positions in each monitoring image sequence; Each monitoring image sequence is compared pairwise. During the comparison process, the spatial distance between the two monitoring image sequences is calculated based on the shooting end position of one monitoring image sequence and the shooting start position of another monitoring image sequence. The spatial distance interval is compared with a set interval distance threshold: If the spatial distance interval is greater than the distance interval threshold, then the two monitoring image sequences are determined to belong to different regions; If the spatial distance interval is less than or equal to the distance interval threshold, then the two monitoring image sequences are determined to belong to the same region; All monitoring image sequences belonging to the same area are stitched together to obtain the monitoring image sequence corresponding to the same area.

[0007] Furthermore, the abnormal operational status information and corresponding monitoring image sequences are input into a pre-trained security detection model, including: The abnormal operating status information and the monitoring image sequence of the corresponding area are time-aligned to obtain the time-aligned status image association information. Obtain preset factory operation standard information; Calculate the similarity between the status image association information and the plant operation standard information, and compare the similarity with a set similarity threshold: If the similarity is less than the similarity threshold, the abnormal operating status information of the device and the monitoring image are input into the pre-trained security detection model.

[0008] Furthermore, calculating the similarity between the status image association information and the plant operation standard information includes: Extract the first semantic feature vector of the state image association information and the second semantic feature vector of the factory operation standard information respectively; Calculate the vector similarity between the first semantic feature vector and the second semantic feature vector, and use the vector similarity as the similarity score.

[0009] Furthermore, the security detection model is pre-trained in the following manner: Obtain a sample dataset of safety hazards in the factory area. The sample dataset of safety hazards in the factory area includes multiple sets of sample data. Each set of sample data contains monitoring image sequence samples, abnormal operating status information samples, and corresponding safety hazard type labels. An initial security detection model was constructed based on ResNet and LSTM networks; Multiple sets of sample data are input into the initial security detection model, which extracts image features from the monitored image sequence samples using a ResNet network and extracts temporal features from the abnormal operating status information samples using an LSTM network. The image features and temporal features are then fused and analyzed to output a predicted security hazard type. Based on the difference between the predicted security hazard type and the corresponding security hazard type label, the network parameters of the initial security detection model are adjusted until the convergence condition is met, resulting in a trained security detection model.

[0010] Furthermore, the safety hazard identification results include safety hazard identification types, and safety inspections are conducted on the target factory area based on the safety hazard identification results, including: Based on the established mapping relationship between the types of security hazards and the security levels, the security level corresponding to the identified security hazard type is obtained; The safety hazard identification results are prioritized according to the safety level, and the target factory area is then subjected to safety inspection based on the ranking results.

[0011] An embodiment of the present invention provides a security detection device based on image processing, comprising: The data acquisition module is used to acquire equipment operating status information of each piece of equipment in the target plant area and each monitoring image sequence; The data segmentation module is used to divide the equipment operating condition information under the same working condition into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment; and to divide each monitoring image sequence to obtain the monitoring image sequence corresponding to each region. The status analysis module is used to input the corresponding equipment operating condition information into the pre-trained working status analysis model for each set of equipment, so that the working status analysis model can analyze and process the equipment operating condition information and output the operating status information of each piece of equipment in the set of equipment. The fault identification module is used to input the abnormal operating status information and the corresponding monitoring image sequence into a pre-trained safety detection model when there is abnormal operating status information. The safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features. The safety detection module is used to conduct safety inspections on the target factory area based on the safety hazard identification results.

[0012] An embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; wherein, when the processor executes the computer program, it implements the image processing-based security detection method according to any one of the present invention.

[0013] One embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to execute the image processing-based security detection method according to any one of the present invention.

[0014] The embodiments of the present invention have the following beneficial effects: This invention provides a security detection method, device, electronic device, and storage medium based on image processing. By inputting equipment operating condition information into a working status analysis model for preprocessing, the method determines whether there are any abnormalities in the equipment operating condition information. If the equipment's operating status information is abnormal, the security detection model's fusion analysis is triggered. This dual confirmation mechanism of equipment abnormality and image abnormality effectively reduces sensor false alarms caused by environmental interference and other factors, improving the accuracy of fault identification. The security detection model extracts image features from the monitored image sequence and temporal features from the abnormal operating status information. After fusing and analyzing the image features and temporal features, the security hazard identification result is output. This deep collaboration between internal equipment parameter monitoring and external visual monitoring overcomes the inherent limitation of sensor-based detection methods that cannot detect abnormal equipment appearance, thus improving the accuracy and reliability of security detection in industrial plants. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a security detection method based on image processing provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the structure of a security detection device based on image processing provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] like Figure 1 As shown, one embodiment of the present invention provides a security detection method based on image processing, which may include at least: S1: Obtain the equipment operating status information of each piece of equipment in the target plant area and the sequence of monitoring images.

[0021] In this embodiment, the factory area monitoring task can be triggered by retrieving monitoring points from a manually set factory area monitoring point database and starting the monitoring equipment deployed at the monitoring points. A complete factory area monitoring cycle is a time range from the start time of monitoring to the end time of monitoring. The start time of the factory area monitoring cycle is determined based on the power-on time of the monitoring equipment, and the end time of the factory area monitoring cycle is determined based on the time when the monitoring equipment completes a full-area scan. Each monitoring image sequence refers to multi-area image data collected within a factory area monitoring cycle. Specifically, it can be multiple image acquisitions within a factory area monitoring cycle, which can be obtained by capturing images through high-definition cameras deployed at various monitoring points in the factory area during the factory area monitoring cycle. Specifically, each monitoring image sequence is obtained by continuously capturing images of key areas, hazardous areas, and areas surrounding equipment within the complete time range from the start time to the end time of the factory area monitoring cycle.

[0022] The equipment operating status information consists of the operating parameter data generated by various production equipment in the factory during a factory monitoring cycle. The operating parameter data specifically includes key equipment operating parameters such as belt speed, belt tension, motor temperature, motor vibration frequency, and motor infrared radiation intensity.

[0023] Operating parameter data can be obtained in real time by sensors deployed on key equipment during the plant monitoring cycle.

[0024] Operational parameter data can also be monitored in real time across the entire plant monitoring cycle to track the operational status of each piece of equipment, and then categorized into the equipment operation data for each time period according to the order of data collection. Each monitoring image sequence can be included within the current plant monitoring cycle, covering the entire area including personnel work areas, equipment operation areas, and hazard warning areas, thereby achieving comprehensive capture of personnel behavior and equipment operating status.

[0025] In this embodiment, the collected monitoring image sequences can refer to image carriers used in the field of industrial plant safety monitoring to capture personnel work status and equipment operation scenarios. These sequences can include various types such as static images and dynamic frame sequences. Specifically, by collecting each monitoring image sequence in real time during daily monitoring activities in the plant area, and subsequently using the monitoring image sequences and equipment operating condition information, accurate detection of safety hazards in the plant area can be achieved.

[0026] In this embodiment, each monitoring image sequence is stored in the form of a high-definition frame sequence, which includes structured fields such as acquisition timestamp, acquisition point identifier, image resolution, and shooting angle.

[0027] S2: Divide the equipment operating condition information under the same working condition into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment; divide each monitoring image sequence to obtain the monitoring image sequence corresponding to each region.

[0028] Furthermore, the equipment operating condition information under the same working condition is divided into the same data group, including: Obtain the operating condition fluctuation coefficient corresponding to the operating condition information of each device. Specifically, after normalizing each parameter in the operating condition information of each group of devices to 0-1, calculate the ratio of the standard deviation of the parameter sequence to the mean to obtain the operating condition fluctuation coefficient of the data group, which is used to characterize the degree of fluctuation in device operation.

[0029] Calculate the difference between the operating condition fluctuation coefficients corresponding to the operating condition information of each pair of devices. The difference reflects the difference in the degree of fluctuation between the operating condition information of the two devices, and can be used to determine whether the operating condition information of the two devices belongs to the same operating condition. Compare the difference with a preset difference threshold. If the difference is less than the set difference threshold, it is determined that the operating conditions of the two devices belong to the same operating condition; If the difference is greater than or equal to the set difference threshold, it is determined that the operating conditions of the two devices do not belong to the same operating condition. The operating condition information of all equipment belonging to the same operating condition is grouped into the same data group.

[0030] In this embodiment, the preset threshold for the difference in equipment operating condition fluctuation coefficient is set to 0.15; that is, when the difference in the fluctuation coefficient between two sets of equipment operating conditions is less than 0.15, they are determined to belong to the same operating condition.

[0031] Furthermore, the step of dividing each monitoring image sequence into monitoring image sequences corresponding to each region includes: Based on the shooting coordinates of the monitoring cameras, obtain the starting and ending shooting positions in each monitoring image sequence.

[0032] Each monitoring image sequence is compared pairwise. During the comparison process, the spatial distance between the two monitoring image sequences is calculated based on the shooting end position of one monitoring image sequence and the shooting start position of the other monitoring image sequence.

[0033] The spatial distance interval is compared with a set interval distance threshold: If the spatial distance interval is greater than the distance interval threshold, then the two monitoring image sequences are determined to belong to different regions; If the spatial distance interval is less than or equal to the distance interval threshold, then the two monitoring image sequences are determined to belong to the same region; All monitoring image sequences belonging to the same area are stitched together to obtain the monitoring image sequence corresponding to the same area.

[0034] In this embodiment, when comparing each pair of monitoring image sequences, if the spatial distance between the shooting end position of one monitoring image sequence and the shooting start position of another monitoring image sequence is greater than a preset distance threshold, it indicates that one monitoring image sequence and the other monitoring image sequence belong to two sets of independent image data from different monitoring areas. Frame segmentation is then performed on the two monitoring image sequences. Frame segmentation specifically refers to splitting one monitoring image sequence and the other monitoring image sequence into multiple independent image frames as monitoring image sequences to be detected. This facilitates subsequent separate detection of images from different monitoring areas, improving the accuracy of factory safety monitoring.

[0035] In this embodiment, when comparing each monitoring image sequence in pairs, if the spatial distance between the shooting end position of one monitoring image sequence and the shooting start position of another monitoring image sequence is less than or equal to a preset distance threshold, it indicates that one monitoring image sequence and the other monitoring image sequence belong to two sets of image data in the same monitoring area. The two monitoring image sequences can be continuously spliced ​​according to the order of shooting positions and merged into a complete monitoring image sequence as the monitoring image sequence to be detected, thereby achieving full coverage detection of the same monitoring area and reducing detection blind spots.

[0036] In this embodiment, the preset interval distance threshold can be preset manually. It can be set manually by referring to the threshold corresponding to the division range of each monitoring area in the factory area. It can be used to distinguish whether two sets of monitoring image sequences belong to the same monitoring area of ​​continuously captured images, and to determine whether the two sets of monitoring image sequences need to be detected separately or spliced.

[0037] In this embodiment, the preset threshold for the interval between the shooting positions of the factory area monitoring images is set to 5 meters; that is, when the shooting positions of two adjacent sets of images are no more than 5 meters apart, they are regarded as the same continuous monitoring area and are stitched together to eliminate the detection blind spot.

[0038] S3: For each set of equipment, the corresponding equipment operating condition information is input into the pre-trained working state analysis model so that the working state analysis model can analyze and process the equipment operating condition information and output the operating state information of each piece of equipment in the set of equipment; the operating state information includes normal operating state information and abnormal operating state information.

[0039] In this embodiment, the preset equipment operating status analysis model can be pre-defined. This model can be a deep learning model based on a combination of CNN and Transformer, pre-trained using industrial equipment operation data training set. Specifically, the model structure of the equipment operating status analysis model includes an input preprocessing layer, a CNN feature extraction branch, a Transformer extraction branch, and a feature fusion and output layer. The input preprocessing layer is used to standardize the input equipment operating condition information. Specifically, it normalizes the operating condition information of each piece of equipment, mapping the values ​​of each parameter to the [0,1] interval to eliminate the influence of differences in the dimensions of different parameters on the model calculation; for operating condition sequences with inconsistent time series lengths, zero-padding is used for alignment, unifying them into a time series window of length 100, and outputting a set of feature sequences.

[0040] The CNN feature extraction branch consists of three cascaded one-dimensional convolutional modules. Its core function is to extract short-term anomalies such as local fluctuations and instantaneous changes in equipment operating parameters, for example, the sudden change in rotational speed at the moment of belt breakage, or the vibration peak when a foreign object intrudes. The configuration of each layer is as follows: The first convolutional module contains a one-dimensional convolutional layer with a kernel size of 7, a stride of 1, and padding of 3, with an output channel count of 32. It is followed by a batch normalization (BatchNorm) layer and a ReLU activation function, and then a one-dimensional max pooling layer with a kernel size of 2 and a stride of 2, which is used to capture the basic fluctuation features of parameters within a short window. The second convolutional module contains a one-dimensional convolutional layer with a kernel size of 5, a stride of 1, and padding of 2, with 64 output channels. It is followed by a batch normalization layer and a ReLU activation function, and then a one-dimensional max pooling layer with a kernel size of 2 and a stride of 2, which is used to extract medium-scale working condition change features. The third convolutional module contains a one-dimensional convolutional layer with a kernel size of 3, a stride of 1, and padding of 1, with 128 output channels. It is followed by a batch normalization layer and a ReLU activation function to capture fine parameter mutation features. The features output by the three convolutional modules are reduced in dimensionality by global average pooling, and finally a local temporal feature vector with a dimension of 128 is obtained.

[0041] The Transformer feature extraction branch and the CNN feature extraction branch are set up in parallel. The Transformer feature extraction branch consists of a positional encoding module and a multi-layer multi-head self-attention encoder. Its core function is to capture global features such as the coupling relationship between various operating parameters and long-term operating condition drift throughout the entire cycle. For example, the fault evolution trend of a motor temperature slowly rising and the vibration frequency rising synchronously is shown in the following module configuration: Learnable positional encoding module: Adds a learnable positional encoding vector with the same length as the sequence to the input feature sequence, embedding temporal positional information into the features, and making up for the self-attention mechanism's deficiency in perceiving temporal positional information.

[0042] Multi-head self-attention encoding layer: A total of two cascaded Transformer encoder layers are set up, each containing eight parallel attention heads with a single attention dimension of 64. The association weight between each temporal sampling point and all sampling points in the entire sequence is calculated through self-attention, modeling the coupling relationship between multiple parameters such as motor temperature, vibration, and speed, as well as the long-term operating condition evolution trend. After the self-attention calculation of each layer, residual connections and layer normalization (LayerNorm) are connected, followed by two layers of feedforward neural network (hidden layer dimension of 256, activation function of GELU), which are also connected with residual connections and layer normalization to ensure the stability of feature propagation.

[0043] Finally, after global average pooling dimensionality reduction, the output is a global dependency feature vector with a dimension of 128.

[0044] The weighted fusion features are sequentially fed into a Dropout layer with a dropout rate of 0.2, a fully connected layer with 64 hidden units (ReLU activation), and finally a fully connected layer with an output dimension of 7. Using Softmax as the activation function, the probability values ​​of each operating state are output, including normal equipment operation and abnormal equipment operation. Abnormal equipment operation includes belt breakage, belt misalignment, foreign object intrusion on the belt, abnormal motor temperature, excessive motor vibration, and abnormal motor infrared radiation. The category with the highest probability value is taken as the final equipment operating state information.

[0045] S4: When there is abnormal operating status information, determine the monitoring area to which the device corresponding to the abnormal operating status information belongs, and obtain the monitoring image sequence corresponding to the monitoring area; input the abnormal operating status information and the corresponding monitoring image sequence into the pre-trained safety detection model, so that the safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features.

[0046] The process of determining the monitoring area to which the device belongs when abnormal operating status information is obtained includes: determining the monitoring area to which the device belongs based on the device installation location and the monitoring image sequence corresponding to each area, and then determining the monitoring image sequence corresponding to the abnormal operating status information.

[0047] Furthermore, the security detection model is pre-trained in the following manner: Obtain a sample dataset of safety hazards in the factory area. The sample dataset of safety hazards in the factory area includes multiple sets of sample data. Each set of sample data contains monitoring image sequence samples, abnormal operating status information samples, and corresponding safety hazard type labels.

[0048] Specifically, 12 months of historical on-site monitoring data from the target factory area were collected, covering monitoring images and equipment operation data under different production periods and environmental conditions. Valid samples were then selected to construct a dataset. Each dataset includes monitoring image sequence samples for the corresponding time window, abnormal operating status information samples, and safety hazard type labels marked by factory safety maintenance professionals. Hazard types cover personnel entering dangerous areas, personnel not wearing safety helmets, belt breakage, belt misalignment, foreign object intrusion into the belt, abnormal motor temperature, excessive motor vibration, and abnormal motor infrared radiation. The dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for parameter iterative updates, the validation set for accuracy verification during training, and the test set for final model performance evaluation. During training, random brightness adjustments, Gaussian noise, and small-angle rotation were applied to the image samples, and random amplitude perturbations and time-series offsets were applied to the operating condition time-series samples to improve the model's environmental robustness and generalization ability.

[0049] An initial security detection model was constructed based on ResNet and LSTM networks.

[0050] Specifically, a dual-branch architecture is used to build the initial model: the visual branch uses ResNet50 as the backbone network, loads pre-trained weights from a general image dataset for initialization, and is responsible for extracting spatial visual features of the image; the temporal branch uses a two-layer stacked bidirectional LSTM network with a hidden layer dimension of 128, and initializes the network weights using an orthogonal initialization method, and is responsible for extracting the temporal evolution features of the device's operating status; at the end of the two branches, a feature fusion layer and a classification output layer are set, and the parameters of the corresponding layers are initialized using a random normal distribution to complete the construction of the initial security detection model.

[0051] Multiple sets of sample data are input into the initial security detection model, which extracts image features from the monitored image sequence samples using a ResNet network and extracts temporal features from the abnormal operating status information samples using an LSTM network. The image features and temporal features are then fused and analyzed to output a predicted security hazard type. Based on the difference between the predicted security hazard type and the corresponding security hazard type label, the network parameters of the initial security detection model are adjusted until the convergence condition is met, resulting in a trained security detection model.

[0052] Specifically, the samples in the training set are input into the initial security detection model in batches, and the forward propagation calculation is completed sequentially: the monitoring image sequence samples are input into the ResNet branch, and the visual features from the bottom layer to the top layer are extracted frame by frame. After inter-frame attention aggregation, the image feature vector is obtained; the abnormal running status information samples are input into the LSTM branch, and the temporal dependence and abnormal evolution features are extracted. After the two types of features are dimensionally aligned and weighted by the feature fusion layer, they are sent to the classification output layer. After the Softmax activation function is applied, the predicted probability of each type of security hazard is output. The category corresponding to the maximum probability is taken as the predicted security hazard type of the sample.

[0053] The model prediction loss is calculated based on the predicted safety hazard type and the corresponding real safety hazard type label for each sample group. A weighted cross-entropy loss function is used, assigning higher loss weights to hazard categories with fewer samples to mitigate identification bias caused by class imbalance. The AdamW optimizer is used to perform backpropagation calculations, updating the model's network parameters layer by layer based on the gradient information of the loss value. During training, a cosine annealing strategy is used to dynamically adjust the learning rate, with an initial learning rate set to 0.0005, which is gradually reduced as training epochs progress to ensure parameter convergence stability in the later stages of training.

[0054] Determine the convergence condition and output the trained model. During training, the classification accuracy and recall rate of the current model for various hazards are verified every 5 rounds using the validation set. When the weighted F1 value of the validation set improves by less than a set condition (e.g., 0.1%) for 15 consecutive rounds, the model is deemed to have met the convergence condition, triggering the early stopping mechanism to terminate training. The model weights of the round with the best performance on the validation set are saved as the final trained safety detection model.

[0055] S5: Conduct safety inspections on the target factory area based on the identified safety hazards.

[0056] In this embodiment, the safety hazard identification result includes the safety hazard identification type, and the safety inspection of the target factory area based on the safety hazard identification result includes: Based on the established mapping relationship between the types of security hazards and the security levels, the security level corresponding to the identified security hazard type is obtained; The safety hazard identification results are prioritized according to the safety level, and the target factory area is then subjected to safety inspection based on the ranking results.

[0057] Specifically, the mapping relationship is defined based on the accident trigger probability, degree of personal injury, scope of production impact, and potential economic loss level corresponding to various types of hidden dangers, setting three safety levels: high-risk, medium-risk, and low-risk. Each identified safety hazard type is matched one by one with the mapping relationship to obtain the initial safety level of the corresponding hazard. In this embodiment, high-risk hazards correspond to belt breakage, personnel entering a dangerous area, and abnormal motor infrared radiation; medium-risk hazards correspond to belt misalignment, foreign object intrusion into the belt, and excessive motor vibration; low-risk hazards correspond to personnel not wearing safety helmets and abnormal motor temperature.

[0058] Subsequently, the hazards are prioritized according to their safety level. The safety level is the primary criterion for prioritization, and all identified safety hazards are arranged in descending order, with high-risk hazards taking precedence over medium-risk hazards, which in turn take precedence over low-risk hazards, thus clarifying the priority of handling. For multiple hazards at the same safety level, the second criterion is the size of the hazard's impact, and the third criterion is the duration of the hazard's duration; hazards with a wider impact and longer duration are prioritized, further refining the order of handling.

[0059] In one embodiment of the present invention, abnormal operating status information and corresponding monitoring image sequences are input into a pre-trained security detection model, including: The abnormal operational status information and the corresponding monitoring image sequence of the area are time-aligned to obtain time-aligned status image association information. Specifically, this includes: A unified time analysis window is defined using the start timestamp of the abnormal operation status information as the alignment start point and the end timestamp as the alignment end point. Extract all image frames that fall within the time window from the monitored image sequence, and extract the parameter sampling sequence within the same time window from the equipment operating condition data to ensure that the time ranges of the two are completely consistent. Linear interpolation is used to resample the operating condition parameter sequence with higher sampling density, so that the time step of the operating condition data is completely matched with the acquisition interval of the image frame. In this embodiment, it is uniformly set to one time step every 2 seconds. The corresponding image frames are bound to the operating parameters one by one according to the time step sequence, with the addition of area identifiers, device identifiers, and timestamp information, and encapsulated into status image association information. In this embodiment, a single set of status image association information contains 20 time steps, corresponding to 20 monitoring images and 20 abnormal operating status information.

[0060] In this embodiment, the preset factory area operation standard information can be manually preset and can refer to the standard content in the same factory area safety management plan. For example, it can be personnel safety operation specifications, equipment normal operation parameter standards, monitoring area safety standards, etc., which can be used to indicate the safety benchmarks that need to be clarified before factory area safety monitoring.

[0061] Calculate the similarity between the status image association information and the plant operation standard information, and compare the similarity with a set similarity threshold: If the similarity is less than a similarity threshold, the abnormal device operation status information and monitoring image sequence are input into a pre-trained security detection model. Specifically, the similarity threshold is set to 0.85. When the similarity between the first semantic feature vector and the second semantic feature vector is greater than 0.85, the system is judged to be in a safe and compliant state, and no further judgment is needed from the subsequent security detection model. When the similarity between the first semantic feature vector and the second semantic feature vector is less than or equal to 0.85, an unsafe or non-compliant situation is judged, and further identification by the subsequent security detection model is required.

[0062] Furthermore, calculating the similarity between the status image association information and the plant operation standard information includes: The first semantic feature vector of the state image association information and the second semantic feature vector of the plant operation standard information are extracted respectively. Specifically, since the state image association information includes operating status information and images, a unified semantic feature vector is generated as the first semantic feature vector using a modal coding and cross-modal fusion method. Twenty monitoring images from the state image association information are sequentially input into a pre-trained ResNet50 visual coding network. The output of the network's global average pooling layer is taken as the visual feature vector of a single frame image. The 20 frame features are weighted and aggregated through an inter-frame attention mechanism to enhance the contribution of frames with significant abnormal features and weaken the interference of frames with no difference. Finally, the image semantic features are obtained to represent the overall visual state of personnel behavior, equipment appearance, and regional environment in the current area. The parameter sequence corresponding to the aligned 20-step abnormal running status information is input into a pre-trained one-dimensional convolutional coding network. After two layers of one-dimensional convolution (with kernel sizes of 5 and 3, and output channels of 64 and 128 respectively) and global average pooling, the running status semantic features are extracted to represent the overall fluctuation level of the current device running parameters and the degree of deviation from the normal range. Image semantic features and runtime semantic features are mapped to the same-dimensional semantic space through two independent fully connected mapping layers. They are then fused using an element-weighted summation method, with the image feature weight set to 0.6 and the runtime semantic feature weight set to 0.4, to obtain the first semantic feature vector.

[0063] Extracting the second semantic feature vector from the factory's operational standards information includes the following process: For the standard subset of the current scene corresponding to the retrieved task, a pre-trained BERT-base Chinese semantic encoding model is used to extract text features; The text content of each job standard is input into the BERT model, and the feature vector corresponding to the label in the model output layer is taken as the semantic feature vector of that standard. If the current scenario corresponds to multiple standard items, the arithmetic mean of the semantic feature vectors of all items is taken to obtain the second semantic feature vector.

[0064] Calculate the vector similarity between the first semantic feature vector and the second semantic feature vector, and use the vector similarity as the similarity score.

[0065] Specifically, cosine similarity is used to calculate the vector similarity between the first semantic feature vector and the second semantic feature vector. The calculation formula is as follows: ; in, This is the first semantic feature vector; The second semantic feature vector; sim is the cosine similarity, with values ​​ranging from... Since both types of features are positive security semantics, the actual similarity values ​​fall within the range of [0,1]. The higher the value, the more compliant the current monitoring status corresponding to the status image association information is.

[0066] In one embodiment of the present invention, for monitoring areas where the equipment is operating normally and there is no abnormal operating status information, a timed inspection trigger mechanism can be set up: at fixed intervals, the monitoring image sequence of each area is retrieved and input into a lightweight personnel violation detection model for initial screening; if the initial screening finds suspected personnel violations, the corresponding information is reported to the plant control center for review and investigation, thereby achieving full coverage detection of personnel violations and equipment failures throughout the plant area.

[0067] Please see Figure 2 Based on the same inventive concept as the above embodiments, one embodiment of the present invention provides a security detection device based on image processing, comprising: Data acquisition module 01 is used to acquire equipment operating status information of each piece of equipment in the target plant area and each monitoring image sequence.

[0068] Specifically, the equipment operating status information of each device consists of the operating parameter data generated by various production equipment in the plant during a plant monitoring cycle. The operating parameter data specifically includes key equipment operating parameters such as belt speed, belt tension, motor temperature, motor vibration frequency, and motor infrared radiation intensity.

[0069] Operating parameter data can be obtained in real time by sensors deployed on key equipment during the plant monitoring cycle.

[0070] It can also monitor the operating status of each piece of equipment in real time within the complete time range of the plant monitoring cycle, and categorize them into the equipment operation data of each time period according to the order of collection. Each monitoring image sequence can be included in the current plant monitoring cycle, covering the entire area of ​​the plant, including personnel work areas, equipment operation areas, and hazard warning areas, thereby achieving comprehensive capture of personnel behavior and equipment operating status.

[0071] In this embodiment, the acquired monitoring image sequences can refer to image carriers used in the field of industrial plant safety monitoring to capture personnel work status and equipment operation scenarios. These sequences can include various types such as static images and dynamic frame sequences. It is understood that the monitoring image sequences can be acquired in real-time during routine monitoring activities in the plant area, and accurate detection of safety hazards in the plant area can be achieved based on the monitoring image sequences and equipment operating condition information.

[0072] The data segmentation module 02 is used to divide the equipment operating condition information under the same working condition into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment; and to divide each monitoring image sequence to obtain the monitoring image sequence corresponding to each region.

[0073] Furthermore, the data segmentation module 02 divides the equipment operating condition information under the same working condition into the same data group, obtaining the equipment operating condition information of several sets of equipment, including: The operating condition fluctuation coefficient for acquiring operating condition information of each device; Calculate the difference between the operating condition fluctuation coefficients corresponding to the operating condition information of each pair of devices, and compare the difference with a preset difference threshold: If the difference is less than the set difference threshold, it is determined that the operating conditions of the two devices belong to the same operating condition; If the difference is greater than or equal to the set difference threshold, it is determined that the operating conditions of the two devices do not belong to the same operating condition. The operating condition information of all equipment belonging to the same operating condition is grouped into the same data group.

[0074] Furthermore, the data segmentation module 02 divides each monitoring image sequence into monitoring images corresponding to each region, including: Based on the shooting coordinates of the monitoring cameras, obtain the starting and ending shooting positions in each monitoring image sequence.

[0075] Each monitoring image sequence is compared pairwise. During the comparison process, the spatial distance between the two monitoring image sequences is calculated based on the shooting end position of one monitoring image sequence and the shooting start position of the other monitoring image sequence.

[0076] The spatial distance interval is compared with a set interval distance threshold: If the spatial distance interval is greater than the distance interval threshold, then the two monitoring image sequences are determined to belong to different regions; If the spatial distance interval is less than or equal to the distance interval threshold, then the two monitoring image sequences are determined to belong to the same region; All monitoring image sequences belonging to the same area are stitched together to obtain the monitoring image sequence corresponding to the same area.

[0077] The status analysis module 03 is used to input the corresponding equipment operating condition information into the pre-trained working status analysis model for each set of equipment, so that the working status analysis model can analyze and process the equipment operating condition information and output the operating status information of each piece of equipment in the set of equipment.

[0078] The fault identification module 04 is used to determine the monitoring area to which the device corresponding to the abnormal operating status information belongs when there is abnormal operating status information, and to obtain the monitoring image sequence corresponding to the monitoring area; input the abnormal operating status information and the corresponding monitoring image sequence into a pre-trained safety detection model, so that the safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features; Safety detection module 05 is used to conduct safety detection on the target factory area based on the safety hazard identification results.

[0079] In one embodiment of the present invention, the fault identification module 04 further includes a model training unit, which is configured to perform the following functions: Obtain a sample dataset of safety hazards in the factory area. The sample dataset of safety hazards in the factory area includes multiple sets of sample data. Each set of sample data contains monitoring image sequence samples, abnormal operating status information samples, and corresponding safety hazard type labels.

[0080] Specifically, 12 months of historical on-site monitoring data from the target factory area were collected, covering monitoring images and equipment operation data under different production periods and environmental conditions. Valid samples were then selected to construct a dataset. Each dataset includes monitoring image sequence samples for the corresponding time window, abnormal operating status information samples, and safety hazard type labels marked by factory safety maintenance professionals. Hazard types cover personnel entering dangerous areas, personnel not wearing safety helmets, belt breakage, belt misalignment, foreign object intrusion into the belt, abnormal motor temperature, excessive motor vibration, and abnormal motor infrared radiation. The dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for parameter iterative updates, the validation set for accuracy verification during training, and the test set for final model performance evaluation. During training, random brightness adjustments, Gaussian noise, and small-angle rotation were applied to the image samples, and random amplitude perturbations and time-series offsets were applied to the operating condition time-series samples to improve the model's environmental robustness and generalization ability.

[0081] An initial security detection model was constructed based on ResNet and LSTM networks.

[0082] Specifically, a dual-branch architecture is used to build the initial model: the visual branch uses ResNet50 as the backbone network, loads pre-trained weights from a general image dataset for initialization, and is responsible for extracting spatial visual features of the image; the temporal branch uses a two-layer stacked bidirectional LSTM network with a hidden layer dimension of 128, and initializes the network weights using an orthogonal initialization method, and is responsible for extracting the temporal evolution features of the device's operating status; at the end of the two branches, a feature fusion layer and a classification output layer are set, and the parameters of the corresponding layers are initialized using a random normal distribution to complete the construction of the initial security detection model.

[0083] Multiple sets of sample data are input into the initial security detection model, which extracts image features from the monitored image sequence samples using a ResNet network and extracts temporal features from the abnormal operating status information samples using an LSTM network. The image features and temporal features are then fused and analyzed to output a predicted security hazard type. Based on the difference between the predicted security hazard type and the corresponding security hazard type label, the network parameters of the initial security detection model are adjusted until the convergence condition is met, resulting in a trained security detection model.

[0084] Specifically, the samples in the training set are input into the initial security detection model in batches, and the forward propagation calculation is completed sequentially: the monitoring image sequence samples are input into the ResNet branch, and the visual features from the bottom layer to the top layer are extracted frame by frame. After inter-frame attention aggregation, the image feature vector is obtained; the abnormal running status information samples are input into the LSTM branch, and the temporal dependence and abnormal evolution features are extracted. After the two types of features are dimensionally aligned and weighted by the feature fusion layer, they are sent to the classification output layer. After the Softmax activation function is applied, the predicted probability of each type of security hazard is output. The category corresponding to the maximum probability is taken as the predicted security hazard type of the sample.

[0085] The model prediction loss is calculated based on the predicted safety hazard type and the corresponding real safety hazard type label for each sample group. A weighted cross-entropy loss function is used, assigning higher loss weights to hazard categories with fewer samples to mitigate identification bias caused by class imbalance. The AdamW optimizer is used to perform backpropagation calculations, updating the model's network parameters layer by layer based on the gradient information of the loss value. During training, a cosine annealing strategy is used to dynamically adjust the learning rate, with an initial learning rate set to 0.0005, which is gradually reduced as training epochs progress to ensure parameter convergence stability in the later stages of training.

[0086] Determine the convergence condition and output the trained model. During training, the classification accuracy and recall rate of the current model for various hazards are verified every 5 rounds using the validation set. When the weighted F1 value of the validation set improves by less than a set condition (e.g., 0.1%) for 15 consecutive rounds, the model is deemed to have met the convergence condition, triggering the early stopping mechanism to terminate training. The model weights of the round with the best performance on the validation set are saved as the final trained safety detection model.

[0087] In one embodiment of the present invention, the fault identification module 04 further includes a time alignment unit, which is configured to perform the following steps: performing time alignment processing on the abnormal operating status information and the monitoring image sequence of the corresponding area to obtain time-aligned status image association information, specifically including: A unified time analysis window is defined using the start timestamp of the abnormal operation status information as the alignment start point and the end timestamp as the alignment end point. Extract all image frames that fall within the time window from the monitored image sequence, and extract the parameter sampling sequence within the same time window from the equipment operating condition data to ensure that the time ranges of the two are completely consistent. Linear interpolation is used to resample the operating condition parameter sequence with higher sampling density, so that the time step of the operating condition data is completely matched with the acquisition interval of the image frame. In this embodiment, it is uniformly set to one time step every 2 seconds. The corresponding image frames are bound to the operating parameters one by one according to the time step sequence, with the addition of area identifiers, device identifiers, and timestamp information, and encapsulated into status image association information. In this embodiment, a single set of status image association information contains 20 time steps, corresponding to 20 monitoring images and 20 abnormal operating status information.

[0088] This invention provides a security detection method, device, electronic device, and storage medium based on image processing. By dividing the operating condition information of various devices into multiple sets of devices according to different operating conditions and dividing the monitoring image sequence according to spatial regions, it achieves structured alignment and preprocessing of two types of heterogeneous data, providing an accurate data foundation for subsequent fusion detection. Based on this, a working state analysis model is first used to perform preliminary analysis of the device operating condition information, filtering out abnormal operating condition information. Then, a security detection model is triggered to perform fusion analysis of the abnormal state information and corresponding image sequences. Through initial screening of abnormal operating conditions and image time sequence correction, only when abnormal sensor parameters and visual manifestations of faults in the monitoring images coexist and corroborate each other can detection be achieved. This reduces the susceptibility to temperature drift and vibration caused by relying solely on sensors for detection. False alarms caused by environmental factors such as dynamic interference or operator errors allow for the accurate identification of genuine safety hazards, preventing real fault signals from being buried among a large number of invalid alarms. Furthermore, the safety detection model extracts image features from the monitoring image sequence and temporal features from abnormal operating status information and performs cross-modal fusion analysis to deeply correlate changes in internal equipment parameters with external visual manifestations. This compensates for the inherent deficiency of single sensors in being unable to detect abnormal equipment appearances (such as belt misalignment, breakage, or foreign object intrusion). At the same time, it achieves equipment condition grouping by determining the pairwise difference of the operating condition fluctuation coefficient and achieves precise region division and stitching of image sequences by determining the spatial distance of the shooting position. This prevents genuine safety hazards from being buried among invalid alarm information, significantly improving the accuracy and reliability of safety hazard identification in the plant area.

[0089] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0090] Those skilled in the art will clearly understand that, for convenience and simplicity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0091] One embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the image processing-based security detection method of any one of the present invention.

[0092] Electronic devices can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. Electronic devices may include, but are not limited to, processors and memory.

[0093] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device through various interfaces and lines.

[0094] Memory can be used to store computer programs. The processor implements various functions of electronic devices by running or executing computer programs stored in memory and accessing data stored in memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc.; the data storage area can store data created based on the use of the mobile phone, etc. In addition, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0095] One embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute the image processing-based security detection method of any one of the present invention.

[0096] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0097] One embodiment of the present invention provides a computer program product that, when run on a terminal device, causes the electronic device to perform the security detection method based on image processing as described above.

[0098] The above are preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A security detection method based on image processing, characterized in that, include: Acquire equipment operating status information and monitoring image sequences for each piece of equipment in the target plant area; The equipment operating condition information under the same working condition is divided into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment. The monitoring image sequences are divided into regions to obtain the monitoring image sequences corresponding to each region; For each set of equipment, the corresponding equipment operating condition information is input into a pre-trained working state analysis model so that the working state analysis model can analyze and process the equipment operating condition information and output the operating state information of each piece of equipment in the set of equipment. When there is abnormal operating status information, the monitoring area to which the device corresponding to the abnormal operating status information belongs is determined, and the monitoring image sequence corresponding to the monitoring area is obtained; the abnormal operating status information and the corresponding monitoring image sequence are input into a pre-trained safety detection model, so that the safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features; Safety inspections are conducted on the target factory area based on the identified safety hazards.

2. The security detection method based on image processing as described in claim 1, characterized in that, Equipment operating condition information under the same operating condition is divided into the same data group, including: The operating condition fluctuation coefficient for acquiring operating condition information of each device; Calculate the difference between the operating condition fluctuation coefficients corresponding to the operating condition information of each pair of devices, and compare the difference with a preset difference threshold: If the difference is less than the set difference threshold, it is determined that the operating conditions of the two devices belong to the same operating condition; If the difference is greater than or equal to the set difference threshold, it is determined that the operating conditions of the two devices do not belong to the same operating condition. The operating condition information of all equipment belonging to the same operating condition is grouped into the same data group.

3. The security detection method based on image processing as described in claim 1, characterized in that, The process of dividing each monitoring image sequence into monitoring image sequences corresponding to each region includes: Obtain the start and end shooting positions in each monitoring image sequence; Each monitoring image sequence is compared pairwise. During the comparison process, the spatial distance between the two monitoring image sequences is calculated based on the shooting end position of one monitoring image sequence and the shooting start position of another monitoring image sequence. The spatial distance interval is compared with a set interval distance threshold: If the spatial distance interval is greater than the distance interval threshold, then the two monitoring image sequences are determined to belong to different regions; If the spatial distance interval is less than or equal to the distance interval threshold, then the two monitoring image sequences are determined to belong to the same region; All monitoring image sequences belonging to the same area are stitched together to obtain the monitoring image sequence corresponding to the same area.

4. The security detection method based on image processing as described in claim 1, characterized in that, The abnormal operating status information and the corresponding monitoring image sequence are input into the pre-trained security detection model, including: The abnormal operating status information and the monitoring image sequence of the corresponding area are time-aligned to obtain the time-aligned status image association information. Obtain preset factory operation standard information; Calculate the similarity between the status image association information and the plant operation standard information, and compare the similarity with a set similarity threshold: If the similarity is less than the similarity threshold, the abnormal operating status information of the device and the monitoring image sequence are input into the pre-trained security detection model.

5. The security detection method based on image processing as described in claim 4, characterized in that, Calculating the similarity between the status image association information and the plant operation standard information includes: Extract the first semantic feature vector of the state image association information and the second semantic feature vector of the factory operation standard information respectively; Calculate the vector similarity between the first semantic feature vector and the second semantic feature vector, and use the vector similarity as the similarity score.

6. The security detection method based on image processing as described in claim 1, characterized in that, The security detection model is pre-trained in the following manner: Obtain a sample dataset of safety hazards in the factory area. The sample dataset of safety hazards in the factory area includes multiple sets of sample data. Each set of sample data contains monitoring image sequence samples, abnormal operating status information samples, and corresponding safety hazard type labels. An initial security detection model was constructed based on ResNet and LSTM networks; Multiple sets of sample data are input into the initial security detection model, which extracts image features from the monitored image sequence samples using a ResNet network and extracts temporal features from the abnormal operating status information samples using an LSTM network. The image features and temporal features are then fused and analyzed to output a predicted security hazard type. Based on the difference between the predicted security hazard type and the corresponding security hazard type label, the network parameters of the initial security detection model are adjusted until the convergence condition is met, resulting in a trained security detection model.

7. The security detection method based on image processing as described in claim 1, characterized in that, The safety hazard identification results include the safety hazard identification type. Based on the safety hazard identification results, safety inspections are conducted on the target factory area, including: Based on the established mapping relationship between the types of security hazards and the security levels, the security level corresponding to the identified security hazard type is obtained; The safety hazard identification results are prioritized according to the safety level, and the target factory area is then subjected to safety inspection based on the ranking results.

8. A security detection device based on image processing, characterized in that, include: The data acquisition module is used to acquire equipment operating status information of each piece of equipment in the target plant area and each monitoring image sequence; The data partitioning module is used to divide the equipment operating condition information under the same working condition into the same data group to obtain the equipment operating condition information corresponding to several sets of equipment. The monitoring image sequences are divided into regions to obtain the monitoring image sequences corresponding to each region; The status parsing module is used to input the corresponding equipment operating condition information into the pre-trained working status parsing model for each set of equipment, so that the working status parsing model can parse and process the equipment operating condition information and output the operating status information of each piece of equipment in the set of equipment. The fault identification module is used to determine the monitoring area to which the device corresponding to the abnormal operating status information belongs when there is abnormal operating status information, and to obtain the monitoring image sequence corresponding to the monitoring area; input the abnormal operating status information and the corresponding monitoring image sequence into a pre-trained safety detection model, so that the safety detection model extracts the image features in the monitoring image sequence and the temporal features in the abnormal operating status information, and outputs the safety hazard identification result after fusing and analyzing the image features and the temporal features; The safety detection module is used to conduct safety inspections on the target factory area based on the safety hazard identification results.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; wherein, when the processor executes the computer program, it implements the image processing-based security detection method as described in any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform the image processing-based security detection method as described in any one of claims 1-7.

Images