A Deep Learning-Based Security Equipment Inspection and Early Warning System and Method

Abstract

This invention discloses a safety equipment inspection and early warning system and method based on deep learning, belonging to the field of intelligent early warning technology. By pre-collecting video data sets and worker historical early warning sets, and based on the video data sets, a wearing detection model, a correctness detection model, and an integrity detection model are constructed and trained respectively. For each worker, the system collects the worker's video to be inspected in real time, determines whether the worker is wearing safety equipment, and if so, determines whether the safety equipment is worn correctly. If correct, it determines whether the worker's safety equipment is complete. If complete, a safety wearing record is generated for the worker. Based on the early warning type, a corresponding early warning record is generated for the worker, the worker's historical early warning set is updated, and a fourth early warning is generated for the worker based on the worker's historical early warning set. This invention significantly improves the intelligent management and control level of the work environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent early warning technology, specifically a security equipment inspection and early warning system and method based on deep learning. Background Technology

[0002] Due to the unique nature of the port production environment, every port worker needs to wear a series of necessary equipment when carrying out production work, among which safety reflective vests and safety helmets play important roles in the port production process.

[0003] Currently, the main method for inspecting safety equipment is manual inspection, which is inefficient and prone to omissions. Therefore, developing a highly efficient and intelligent inspection and alarm system is of great significance in addressing the aforementioned methods for inspecting safety equipment used by port workers.

[0004] Patent application CN117789407A discloses a high-altitude operation safety management system and method based on unmanned aerial vehicles (UAVs). This method uses a thermal imaging sensing system on the UAV to obtain the operating height of the worker and controls the UAV to hover at that height. It also uses cameras on the UAV to acquire images of the worker's equipment installation. Based on these images and an equipment installation detection model, the method detects the installation status of the worker's equipment. The UAV acquires real-time images of the worker's equipment installation and uses the equipment installation detection model to determine whether the worker's safety belt is being worn correctly. However, this method only detects whether the worker is wearing safety equipment correctly and fails to address the issue of whether the equipment is complete.

[0005] Therefore, this invention proposes a security equipment inspection and early warning system and method based on deep learning. Summary of the Invention

[0006] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention proposes a safety equipment inspection and early warning system and method based on deep learning, which significantly improves the intelligent management and control level of the working environment.

[0007] To achieve the above objectives, a deep learning-based security equipment inspection and early warning method is proposed, comprising the following steps:

[0008] Step 1: Pre-collect video data sets and historical worker warning sets;

[0009] Step 2: Based on the video dataset, construct and train the wearing detection model, the correctness detection model, and the integrity detection model respectively;

[0010] Step 3: For each worker, collect the video of the worker to be inspected in real time. Based on the video and the wearing detection model, determine whether the worker is wearing safety equipment. If so, proceed to Step 4; if not, issue the first warning and proceed to Step 6.

[0011] Step 4: Based on the video to be inspected and the correct detection model, determine whether the worker is wearing safety equipment correctly. If correct, proceed to Step 5; if incorrect, issue a second warning and proceed to Step 6.

[0012] Step 5: Based on the video to be inspected and the integrity detection model, determine whether the safety equipment worn by the worker is complete. If complete, generate a safety wearing record for the worker; if incomplete, issue a third warning and proceed to step 6.

[0013] Step 6: Based on the warning type, generate a corresponding warning record for the worker, update the worker's historical warning set, and generate a fourth warning for the worker based on the worker's historical warning set.

[0014] The video data set is collected in the following way:

[0015] By installing video capture equipment at the workplace, images of workers can be captured at different times, in different environments, and for different types of work, and the workers' safety equipment wearing conditions can vary.

[0016] Label each worker's image with tags and equipment frames;

[0017] The equipment frame is defined as a rectangular frame that covers the safety equipment in each frame of an image of a worker wearing safety equipment.

[0018] For each image of a worker wearing a tag indicating that the tag has been worn, label it with the correctness of the tag.

[0019] For each image of a worker wearing the correct identification tag, an integrity tag is added.

[0020] All worker images, after being fully annotated, constitute a video data set.

[0021] The collection method for the worker historical early warning set is as follows:

[0022] The basic information of each worker and the historical early warning records of each worker are read from the existing data storage devices in the workplace to form a worker historical early warning set.

[0023] The method for constructing and training the wear detection model is as follows:

[0024] Choose any target detection model as the wear detection model;

[0025] Each worker image is divided according to the tag worn. Worker images with a tag of 0 are represented as images without a tag, and worker images with a tag of 1 are represented as images with a tag.

[0026] The wearing label for each frame of the unworn image is marked as 0, and the wearing label for each frame of the worn image is marked as 1;

[0027] The image sample set is composed of all frames of images of all workers who are labeled with tags.

[0028] The image sample set is divided into a first training set, a first validation set, and a first test set.

[0029] Load the pre-trained weights and initialize the parameters of the wear detection model;

[0030] The error of the wear detection model is evaluated by using a comprehensive loss function that includes location loss, confidence loss, and category loss as the loss function of the wear detection model.

[0031] The wear detection model is trained on the first training set. The loss function is minimized by iteratively optimizing the model parameters. Optimizers, including Adam or SGD, are used to update the weights of the wear detection model.

[0032] During the training of the wear detection model, the validation performance of the wear detection model is evaluated using the first validation set, and the hyperparameters of the wear detection model are adjusted based on the validation results of the first validation set.

[0033] The wear detection model was tested on the first test set to verify its performance on unseen data.

[0034] The correct detection model is constructed and trained as follows:

[0035] All images of the wearer were selected as the first training sample set for the correct detection model.

[0036] Extract equipment temporal features from each worker image in the first training sample set;

[0037] Use the temporal features of all worn equipment in the images as the correct feature sample set;

[0038] The set of correct feature samples is divided into a second training set, a second validation set, and a second test set.

[0039] Choose any time series analysis model as the correct detection model;

[0040] The binary cross-entropy loss function between the correctness label and the correctness prediction output by the correct detection model is used as the loss function for the correct detection model.

[0041] Input all the equipment time-series features from the second training set into the correct detection model to train the correct detection model, and use the correctness label to supervise the training process of the correct detection model.

[0042] The performance of the correct detection model is evaluated on the second validation set, and the parameters of the correct detection model are adjusted to improve accuracy. Based on the validation results of the second validation set, the hyperparameters of the correct detection model are adjusted.

[0043] The correct detection model was evaluated using the second test set.

[0044] The integrity detection model is constructed and trained as follows:

[0045] All integrity-labeled worker images were selected as the second training sample set for the integrity detection model.

[0046] The images of all frames in each worker image segment in the second training sample set are extracted to form the second image sample set. Each frame image is assigned a corresponding integrity label, and the integrity label of each frame image is consistent with the integrity label of the corresponding worker image.

[0047] For each frame of the second image sample set, edge detection technology is used to identify the edge shape of the security equipment.

[0048] For each frame in the second image sample set, texture changes on the surface of the safety equipment are extracted using texture analysis techniques.

[0049] For each frame in the second image sample set, the color consistency of the safety equipment surface is detected by using color consistency detection technology.

[0050] For each worker image in the second training sample set, the edge morphology, texture variation, and color consistency of each frame are extracted, and edge morphology sequence, texture variation sequence, and color consistency sequence are generated in the order of video frames to form an integrity feature sequence.

[0051] The complete feature sample set is composed of the complete feature sequence of all second image sample sets;

[0052] The complete feature sample set is divided into a third training set, a third validation set, and a third test set;

[0053] Choose any time series analysis model as the integrity detection model;

[0054] The binary cross-entropy loss function between the integrity label and the integrity prediction value output by the integrity detection model is used as the loss function of the integrity detection model.

[0055] Input all integrity feature sequences from the third training set into the integrity detection model to train the integrity detection model, and use integrity labels to supervise the training process of the integrity detection model;

[0056] The performance of the integrity detection model is evaluated on the third validation set, and the model parameters are adjusted to improve accuracy. The hyperparameters of the integrity detection model are then adjusted based on the validation results of the third validation set.

[0057] The integrity detection model was evaluated using the third test set.

[0058] The method for determining whether a worker is wearing safety equipment based on the video to be inspected and the wearing detection model is as follows:

[0059] Each frame of the video to be inspected is extracted and input into the wearing detection model. If the output of the wearing detection model for at least N frames indicates that the device is being worn, then it is determined to be wearing the device; otherwise, it is determined to be not wearing the device. N is a preset quantity parameter.

[0060] The method for generating a corresponding early warning record for the worker based on the early warning type and updating the worker's historical early warning set is as follows:

[0061] By matching the worker's facial data in the video to be inspected with the baseline facial images in the worker's historical warning set, the worker's basic information can be obtained.

[0062] Add a new warning record for this worker to the worker's historical warning set.

[0063] The method for generating a fourth warning for a worker based on the worker's historical warning set is as follows:

[0064] Find all historical warning records for the worker in the worker's historical warning set;

[0065] Calculate the average frequency of each type of warning in the historical warning record, as well as the average frequency of all warnings.

[0066] Preset the frequency threshold for each type of warning and the total frequency threshold;

[0067] If the average frequency of any warning type exceeds the corresponding frequency threshold, or the average frequency of all warnings exceeds the total frequency threshold, a fourth warning will be issued using the preset warning method.

[0068] A deep learning-based security equipment inspection and early warning system is proposed, comprising a historical data collection module, a model training module, a compliance detection module, and an early warning module; wherein the modules are electrically connected to each other.

[0069] The historical data collection module collects video data sets and worker historical early warning sets in advance, and sends the video data sets to the model training module and the worker historical early warning sets to the early warning module.

[0070] The model training module constructs and trains the wearing detection model, the correctness detection model, and the integrity detection model based on the video dataset, and then sends the wearing detection model, the correctness detection model, and the integrity detection model to the compliance detection module.

[0071] The compliance detection module collects real-time video footage of each worker. Based on the video and the wearing detection model, it determines whether the worker is wearing safety equipment. If not, a first warning is sent to the warning module. If the worker is wearing the equipment, based on the video and the correctness detection model, it determines whether the worker is wearing the safety equipment correctly. If not, a second warning is sent to the warning module. If the equipment is correctly worn, based on the video and the integrity detection model, it determines whether the worker is wearing the safety equipment completely. If complete, a safety wearing record is generated for the worker; if incomplete, a third warning is sent to the warning module.

[0072] The early warning module generates a corresponding early warning record for the worker based on the early warning type, updates the worker's historical early warning set, and generates a fourth early warning for the worker based on the worker's historical early warning set.

[0073] Compared with the prior art, the beneficial effects of the present invention are:

[0074] This invention pre-collects video datasets and historical worker warning datasets. Based on the video datasets, it constructs and trains wearing detection models, correctness detection models, and integrity detection models. For each worker, the invention collects their video feed in real time. Based on the video feed and the wearing detection model, it determines whether the worker is wearing safety equipment. If so, it determines whether the equipment is worn correctly, and if so, whether the equipment is complete. If complete, it generates a safety wearing record for the worker. Finally, based on the warning type, it generates a corresponding warning record for the worker, updates the historical worker warning dataset, and generates a fourth warning for the worker based on the historical warning dataset. By collecting video datasets and historical worker warning data, the wearing detection model, correctness detection model, and integrity detection model are constructed and trained sequentially. In practical applications, the invention acquires the worker's video stream in real time. First, it uses the wearing detection model to determine whether the worker is wearing safety equipment. If so, it uses the correctness detection model to determine whether the equipment is worn correctly, and if so, it uses the integrity detection model to determine whether the equipment is complete. Based on the detection results, different types of early warning records are generated. These records are then combined with the worker's historical early warning records to comprehensively assess the early warning level and generate a corresponding fourth early warning. This solution sequentially achieves intelligent detection of the wearing, correctness, and integrity of safety equipment, providing an efficient and reliable technical approach to improve worker safety and significantly enhancing the level of intelligent management of the work environment. Attached Figure Description

[0075] Figure 1 This is a flowchart of a deep learning-based security equipment inspection and early warning method according to Embodiment 1 of the present invention;

[0076] Figure 2 This is a module connection diagram of a deep learning-based security equipment inspection and early warning system in Embodiment 2 of the present invention. Detailed Implementation

[0077] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0078] like Figure 1 As shown, a deep learning-based security equipment inspection and early warning method includes the following steps:

[0079] Step 1: Pre-collect video data sets and historical worker warning sets;

[0080] Step 2: Based on the video dataset, construct and train the wearing detection model, the correctness detection model, and the integrity detection model respectively;

[0081] Step 3: For each worker, collect the video of the worker to be inspected in real time. Based on the video and the wearing detection model, determine whether the worker is wearing safety equipment. If so, proceed to Step 4; if not, issue the first warning and proceed to Step 6.

[0082] Step 4: Based on the video to be inspected and the correct detection model, determine whether the worker is wearing safety equipment correctly. If correct, proceed to Step 5; if incorrect, issue a second warning and proceed to Step 6.

[0083] Step 5: Based on the video to be inspected and the integrity detection model, determine whether the safety equipment worn by the worker is complete. If complete, generate a safety wearing record for the worker; if incomplete, issue a third warning and proceed to step 6.

[0084] Step 6: Based on the warning type, generate a corresponding warning record for the worker, update the worker's historical warning set, and generate a fourth warning for the worker based on the worker's historical warning set.

[0085] Understandably, in order to build a judgment model that can identify whether workers are wearing safety equipment safely through video, it is necessary to first collect sufficient training data, that is, to first collect a set of video data.

[0086] Specifically, the video data set is collected in the following way:

[0087] By installing video capture equipment at the workplace, images of workers can be captured at different times, in different environments, and for different types of work, and the workers' safety equipment wearing conditions can vary.

[0088] Each worker image is labeled with a tag and equipment frame; the tag may include 0 and 1, where 0 indicates no tag is worn and 1 indicates that the tag is worn.

[0089] The equipment frame is defined as a rectangular frame that covers the safety equipment in each frame of an image of a worker wearing safety equipment.

[0090] Each worker's video image, indicating that the safety equipment is being worn, is labeled with a correctness tag. The correctness tag includes either 0 or 1, where 0 indicates incorrect wearing and 1 indicates correct wearing. Preferably, when labeling the correctness, attention is paid to the position and stability of the safety equipment in each frame of the video. For example, the safety helmet should always cover the correct area of ​​the head and should not deviate or slip off.

[0091] Each correctness label indicates that the worker's safety equipment is worn correctly, and an integrity label is added; the integrity label includes 0 or 1, where 0 indicates that the safety equipment is not complete, and 1 indicates that the safety equipment is complete.

[0092] All worker images, after being fully annotated, constitute a video data set.

[0093] Specifically, the time period, environment, and job type are set according to the actual requirements of the workplace;

[0094] The differences in the wearing of safety equipment by workers refer to situations such as workers wearing safety equipment correctly, not wearing safety equipment, wearing safety equipment incorrectly, and wearing incomplete safety equipment.

[0095] Furthermore, the collection method for the worker historical early warning set is as follows:

[0096] The basic information of each worker and the historical early warning records of each worker are read from the existing data storage devices in the workplace to form a worker historical early warning set.

[0097] Specifically, the worker's basic information includes name, employee number, job type, and baseline facial image, etc.

[0098] The historical warning records include the time of each warning, the type of warning (not wearing, wearing incorrectly, incomplete equipment), and the result of the handling.

[0099] Furthermore, the construction and training method of the wear detection model is as follows:

[0100] Any object detection model can be selected as the wearable detection model; specifically, the object detection model includes RCNN model, YOLO model, etc.

[0101] Each worker image is divided according to the tag worn. Worker images with a tag of 0 are represented as images without a tag, and worker images with a tag of 1 are represented as images with a tag.

[0102] The wearing label for each frame of the unworn image is marked as 0, and the wearing label for each frame of the worn image is marked as 1;

[0103] The image sample set is composed of all frames of images of all workers who are labeled with tags.

[0104] The image sample set is divided into a first training set, a first validation set, and a first test set. Generally, the first training set, the first validation set, and the first test set are divided in a ratio of 70%, 20%, and 10%, respectively, to ensure that the wear detection model has sufficient data for learning during the training phase and has an independent dataset for validation and testing.

[0105] Preferably, before training the wear detection model, data augmentation techniques can be applied to the first training set to improve the model's robustness. For example, images can be rotated, scaled, flipped, and their colors adjusted to make the wear detection model more adaptable to different wearing scenarios and lighting conditions.

[0106] Load pre-trained weights to initialize the parameters of the wear detection model;

[0107] A comprehensive loss function, including location loss, confidence loss, and category loss, is used as the loss function for the wear detection model to evaluate the prediction error of the wear detection model.

[0108] The wear detection model is trained on the first training set. The loss function is minimized by iteratively optimizing the model parameters. Optimizers, including Adam or SGD, are used to update the weights of the wear detection model.

[0109] During the training of the wear detection model, the validation performance of the wear detection model is evaluated using a first validation set. The validation performance can be measured by metrics such as mAP. Based on the validation results of the first validation set, the hyperparameters of the wear detection model are adjusted to improve its performance.

[0110] The wear detection model was tested on the first test set to verify its performance on unseen data. The performance evaluation metrics included precision, recall, and F1 score to comprehensively measure the model's detection capability.

[0111] Furthermore, the correct detection model is constructed and trained as follows:

[0112] All images of the wearer were selected as the first training sample set for the correct detection model.

[0113] Extract equipment temporal features from each worker image in the first training sample set;

[0114] Specifically, the equipment temporal features are obtained by using optical flow technology to capture the displacement and dynamic changes of the equipment frame of the safety equipment between video frames, and by using background subtraction and edge detection technology to obtain the position and boundary changes of the safety equipment in each frame of the wearing image.

[0115] It should be noted that optical flow is a vector field used to represent the motion of pixels in an image across time frames. It reflects the displacement and dynamic changes of an object in consecutive frames. The optical flow field typically consists of two components: horizontal flow and vertical flow. The displacement of each pixel between consecutive frames can be represented by a vector, and this vector representation can serve as a temporal feature of equipment. By calculating optical flow, the model can capture the positional changes of safety equipment, such as a safety helmet, during worker movements, which helps to identify whether it is worn correctly.

[0116] The position and boundary changes of the bounding box can be collected by collecting the size and center point position of the bounding box in each frame, and constructing a time series of the size and center point position of the bounding box in the order of each frame, thus forming another equipment timing feature.

[0117] Use the temporal features of all worn equipment in the images as the correct feature sample set;

[0118] The set of correct feature samples is divided into a second training set, a second validation set, and a second test set.

[0119] Any time series analysis model can be selected as the correct detection model; the time series analysis model includes LSTM model, a combination of LSTM model and CNN model, etc.

[0120] The binary cross-entropy loss function between the correctness label and the correctness prediction value output by the correctness detection model is used as the loss function of the correctness detection model; the correctness prediction value is 1 or 0, which respectively indicate that the worker is correctly wearing safety equipment or not correctly wearing safety equipment as determined by the correctness detection model.

[0121] The temporal features of all equipment in the second training set are input into the correct detection model to train the correct detection model. The training process of the correct detection model is supervised by the correctness label. In this way, the correct detection model can learn the changes in wearing state under complex actions by inputting sequence samples.

[0122] The performance of the correct detection model is evaluated on the second validation set, and the parameters of the correct detection model are adjusted to improve accuracy. Based on the validation results of the second validation set, the hyperparameters of the correct detection model, such as learning rate and batch size, are adjusted to ensure the generalization ability of the correct detection model on unseen data.

[0123] The second test set was used to evaluate the correct detection model to ensure that the correct detection model could accurately determine the wearing status of safety equipment in the video. The evaluation metrics for the second test set included accuracy, precision, recall, and F1 score.

[0124] Furthermore, the integrity detection model is constructed and trained as follows:

[0125] All integrity-labeled worker images were selected as the second training sample set for the integrity detection model.

[0126] The images of all frames in each worker image segment in the second training sample set are extracted to form the second image sample set. Each frame image is assigned a corresponding integrity label, and the integrity label of each frame image is consistent with the integrity label of the corresponding worker image.

[0127] For each frame of the second image sample set, edge detection technology is used to identify the edge shape of the safety equipment in order to identify whether there are abnormalities such as cracks or damage in the safety equipment.

[0128] For each frame in the second image sample set, texture changes on the surface of the safety equipment are extracted using texture analysis techniques to identify wear or material aging of the safety equipment.

[0129] For each frame of the second image sample set, the color consistency of the safety equipment surface is detected by using color consistency detection technology to determine whether there is fading or contamination on the safety equipment surface.

[0130] For each worker image in the second training sample set, the edge morphology, texture variation, and color consistency of each frame are extracted, thereby generating an edge morphology sequence, texture variation sequence, and color consistency sequence in the order of video frames, which together form an integrity feature sequence.

[0131] The complete feature sample set is composed of the complete feature sequence of all second image sample sets;

[0132] The complete feature sample set is divided into a third training set, a third validation set, and a third test set;

[0133] Choose any time series analysis model as the integrity detection model;

[0134] The binary cross-entropy loss function between the integrity label and the integrity prediction value output by the integrity detection model is used as the loss function of the integrity detection model; the positive integrity prediction value is 1 or 0, which respectively indicate whether the safety equipment worn by the worker is complete or incomplete as determined by the integrity detection model.

[0135] All integrity feature sequences from the third training set are input into the integrity detection model for training. The training process of the integrity detection model is supervised by integrity labels. This enables the integrity detection model to learn the performance of wearing integrity under complex actions by input sequence samples.

[0136] The performance of the integrity detection model is evaluated on the third validation set, and the model parameters are adjusted to improve accuracy. The hyperparameters of the integrity detection model are then adjusted based on the validation results of the third validation set.

[0137] The integrity detection model was evaluated using a third test set to ensure that it could accurately determine the integrity of security equipment in the video.

[0138] The real-time collection of the worker's video to be inspected means capturing the worker's real-time image through a video capture device when the worker enters the work site, and using it as the video to be inspected.

[0139] The method for determining whether a worker is wearing safety equipment based on the video to be inspected and the wearing detection model is as follows:

[0140] Each frame of the video to be inspected is extracted and input into the wearing detection model. If the output of the wearing detection model for at least N frames indicates that the device is being worn, then it is determined to be wearing the device; otherwise, it is determined to be not wearing the device. N is a preset quantity parameter.

[0141] The method for determining whether a worker is wearing safety equipment correctly, based on the video to be inspected and the correct detection model, is as follows:

[0142] Extract the equipment temporal features from the video to be inspected, and input the equipment temporal features from the video to be inspected into the correct detection model to obtain the judgment result of the correctness prediction value output by the correct detection model.

[0143] The method for determining whether the worker's safety equipment is intact based on the video to be inspected and the integrity detection model is as follows:

[0144] Extract the integrity feature sequence of the video to be inspected, and input the integrity feature sequence of the video to be inspected into the integrity detection model to obtain the judgment result of the integrity prediction value output by the integrity detection model.

[0145] The method for generating a corresponding early warning record for the worker based on the early warning type and updating the worker's historical early warning set is as follows:

[0146] By matching the worker's facial data in the video to be inspected with the baseline facial images in the worker's historical warning set, the worker's basic information can be obtained.

[0147] Add a new warning record for the worker to the worker's historical warning set, including the worker's name, employee number, job type, warning time, and warning type.

[0148] The methods for issuing the first warning, the second warning, and the third warning can each be preset to issue warnings for the worker according to the corresponding warning type.

[0149] The preset warning methods include public or non-public warning methods such as voice broadcasting and screen display.

[0150] The method for generating a fourth warning for the worker based on the worker's historical warning set is as follows:

[0151] Find all historical warning records for the worker in the worker's historical warning set;

[0152] Calculate the average frequency of each type of warning in the historical warning record, as well as the average frequency of all warnings.

[0153] Preset the frequency threshold for each type of warning and the total frequency threshold;

[0154] If the average frequency of any warning type exceeds the corresponding frequency threshold, or the average frequency of all warnings exceeds the total frequency threshold, a fourth warning will be issued using the preset warning method. Example

[0155] like Figure 2 As shown, a security equipment inspection and early warning system based on deep learning includes a historical data collection module, a model training module, a compliance detection module, and an early warning module; wherein, the modules are connected to each other electrically.

[0156] The historical data collection module collects video data sets and worker historical early warning sets in advance, and sends the video data sets to the model training module and the worker historical early warning sets to the early warning module.

[0157] The model training module constructs and trains the wearing detection model, the correctness detection model, and the integrity detection model based on the video dataset, and then sends the wearing detection model, the correctness detection model, and the integrity detection model to the compliance detection module.

[0158] The compliance detection module collects real-time video footage of each worker. Based on the video and the wearing detection model, it determines whether the worker is wearing safety equipment. If not, a first warning is sent to the warning module. If the worker is wearing the equipment, based on the video and the correctness detection model, it determines whether the worker is wearing the safety equipment correctly. If not, a second warning is sent to the warning module. If the equipment is correctly worn, based on the video and the integrity detection model, it determines whether the worker is wearing the safety equipment completely. If complete, a safety wearing record is generated for the worker; if incomplete, a third warning is sent to the warning module.

[0159] The early warning module generates a corresponding early warning record for the worker based on the early warning type, updates the worker's historical early warning set, and generates a fourth early warning for the worker based on the worker's historical early warning set.

[0160] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0161] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0162] The preset parameters or preset thresholds mentioned above are all set by those skilled in the art based on actual conditions or obtained through large-scale data simulation.

[0163] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

[0164] It should be noted that the collection and use of all personal information involved in this invention were carried out only after obtaining the explicit authorization and separate consent of the relevant workers. Specifically, the collection, storage, transmission, and use of workers' basic information (including but not limited to name, employee number, job type, and baseline facial image) and sensitive personal information such as facial data obtained during the actual testing process strictly comply with the provisions of relevant national laws and regulations.

Claims

1. A security equipment inspection and early warning method based on deep learning, characterized in that, Includes the following steps: Step 1: Pre-collect video data sets and historical worker warning sets; Step 2: Based on the video dataset, construct and train the wearing detection model, the correctness detection model, and the integrity detection model respectively; The correct detection model is constructed and trained in the following way: All images of the wearer were selected as the first training sample set for the correct detection model. Equipment temporal features are extracted from each worker image in the first training sample set. These features are obtained by capturing the displacement and dynamic changes of the safety equipment frame between video frames using optical flow technology, and by using background subtraction and edge detection techniques to obtain the position and boundary changes of the safety equipment in each frame of the wearing image. The optical flow field consists of two components: horizontal flow and vertical flow. The displacement of each pixel between consecutive frames is represented by a vector, which serves as one equipment temporal feature. The position and boundary changes of the equipment frame are obtained by collecting the size and center point position of the equipment frame in each frame and constructing a time series of the size and center point position of the equipment frame in the order of each frame, thus forming another equipment temporal feature. Use the temporal features of all worn equipment in the images as the correct feature sample set; The set of correct feature samples is divided into a second training set, a second validation set, and a second test set. Choose any time series analysis model as the correct detection model; The binary cross-entropy loss function between the correctness label and the correctness prediction output by the correct detection model is used as the loss function for the correct detection model. Input all the equipment time-series features from the second training set into the correct detection model to train the correct detection model, and use the correctness label to supervise the training process of the correct detection model. Evaluate the performance of the correct detection model on the second validation set and adjust the parameters of the correct detection model; adjust the hyperparameters of the correct detection model based on the validation results of the second validation set. The correct detection model was evaluated using the second test set; When labeling for accuracy, pay attention to the position and stability of the safety equipment in each frame of the video; Step 3: For each worker, collect the video of the worker to be inspected in real time. Based on the video and the wearing detection model, determine whether the worker is wearing safety equipment. If so, proceed to Step 4; if not, issue the first warning and proceed to Step 6. Step 4: Based on the video to be inspected and the correct detection model, determine whether the worker is wearing safety equipment correctly. If correct, proceed to Step 5; if incorrect, issue a second warning and proceed to Step 6. Step 5: Based on the video to be inspected and the integrity detection model, determine whether the safety equipment worn by the worker is complete. If complete, generate a safety wearing record for the worker; if incomplete, issue a third warning and proceed to step 6. Step 6: Based on the warning type, generate a corresponding warning record for the worker, update the worker's historical warning set, and generate a fourth warning for the worker based on the worker's historical warning set.

2. The method for security equipment inspection and early warning based on deep learning according to claim 1, characterized in that, The video data set is collected in the following way: By installing video capture equipment at the workplace, images of workers can be captured at different times, in different environments, and for different types of work, and the workers' safety equipment wearing conditions can vary. Label each worker's image with tags and equipment frames; The equipment frame is defined as a rectangular frame that covers the safety equipment in each frame of an image of a worker wearing safety equipment. For each image of a worker wearing a tag indicating that the tag has been worn, label it with the correctness of the tag. For each image of a worker wearing the correct identification tag, an integrity tag is added. All worker images, after being fully annotated, constitute a video data set.

3. The method for security equipment inspection and early warning based on deep learning according to claim 2, characterized in that, The collection method for the worker historical early warning set is as follows: The basic information of each worker and the historical early warning records of each worker are read from the existing data storage devices in the workplace to form a worker historical early warning set.

4. The method for security equipment inspection and early warning based on deep learning according to claim 3, characterized in that, The method for constructing and training the wear detection model is as follows: Choose any target detection model as the wear detection model; Each worker image is divided according to the tag worn. Worker images with a tag of 0 are represented as images without a tag, and worker images with a tag of 1 are represented as images with a tag. The wearing label for each frame of the unworn image is marked as 0, and the wearing label for each frame of the worn image is marked as 1; The image sample set is composed of all frames of images of all workers who are labeled with tags. The image sample set is divided into a first training set, a first validation set, and a first test set. Load the pre-trained weights and initialize the parameters of the wear detection model; The error of the wear detection model is evaluated by using a comprehensive loss function that includes location loss, confidence loss, and category loss as the loss function of the wear detection model. The wear detection model was trained on the first training set, and the loss was minimized by iteratively optimizing the model parameters. The function uses optimizers, including Adam or SGD, to update the weights of the wear detection model; During the training of the wear detection model, the validation performance of the wear detection model is evaluated using the first validation set, based on... Based on the validation results of the first validation set, adjust the hyperparameters of the wear detection model; The wear detection model was tested on the first test set to verify its performance on unseen data.

5. The method for security equipment inspection and early warning based on deep learning according to claim 4, characterized in that, The integrity detection model is constructed and trained as follows: All integrity-labeled worker images were selected as the second training sample set for the integrity detection model. The images of all frames in each worker image segment in the second training sample set are extracted to form the second image sample set. Each frame image is assigned a corresponding integrity label, and the integrity label of each frame image is consistent with the integrity label of the corresponding worker image. For each frame of the second image sample set, edge detection technology is used to identify the edge shape of the security equipment. For each frame in the second image sample set, texture changes on the surface of the safety equipment are extracted using texture analysis techniques. For each frame in the second image sample set, the color consistency of the safety equipment surface is detected by using color consistency detection technology. For each worker image in the second training sample set, the edge morphology, texture variation, and color consistency of each frame are extracted, and edge morphology sequence, texture variation sequence, and color consistency sequence are generated in the order of video frames to form an integrity feature sequence. The complete feature sample set is composed of the complete feature sequence of all second image sample sets; The complete feature sample set is divided into a third training set, a third validation set, and a third test set; Choose any time series analysis model as the integrity detection model; The binary cross-entropy loss function is used to compare the integrity predictions output by the integrity label and integrity detection models. As the loss function of the integrity detection model; Input all integrity feature sequences from the third training set into the integrity detection model to train the integrity detection model, and use integrity labels to supervise the training process of the integrity detection model; The performance of the integrity detection model is evaluated on the third validation set, and the parameters of the integrity detection model are adjusted. Based on the validation results of the third validation set, the hyperparameters of the integrity detection model are adjusted. The integrity detection model was evaluated using the third test set.

6. The method for security equipment inspection and early warning based on deep learning according to claim 5, characterized in that, The method for determining whether a worker is wearing safety equipment based on the video to be inspected and the wearing detection model is as follows: Each frame of the video to be inspected is extracted and input into the wearing detection model. If the output of the wearing detection model for at least N frames indicates that the device is being worn, then it is determined to be wearing the device; otherwise, it is determined to be not wearing the device. N is a preset quantity parameter.

7. The method for security equipment inspection and early warning based on deep learning according to claim 6, characterized in that, The method for generating a corresponding early warning record for the worker based on the early warning type and updating the worker's historical early warning set is as follows: By matching the worker's facial data in the video to be inspected with the baseline facial images in the worker's historical warning set, the worker's basic information can be obtained. Add a new warning record for this worker to the worker's historical warning set.

8. The method for security equipment inspection and early warning based on deep learning according to claim 7, characterized in that, The method for generating a fourth warning for a worker based on the worker's historical warning set is as follows: Find all historical warning records for the worker in the worker's historical warning set; Calculate the average frequency of each type of warning in the historical warning record, as well as the average frequency of all warnings. Preset the frequency threshold for each type of warning and the total frequency threshold; If the average frequency of any warning type exceeds the corresponding frequency threshold, or the average frequency of all warnings exceeds the total frequency threshold, a fourth warning will be issued using the preset warning method.

9. A deep learning-based security equipment inspection and early warning system, used to implement the deep learning-based security equipment inspection and early warning method according to any one of claims 1-8, characterized in that, It includes a historical data collection module, a model training module, a compliance detection module, and an early warning module; the modules are connected electrically. The historical data collection module collects video data sets and worker historical early warning sets in advance, and sends the video data sets to the model training module and the worker historical early warning sets to the early warning module. The model training module constructs and trains the wearing detection model, the correctness detection model, and the integrity detection model based on the video dataset, and then sends the wearing detection model, the correctness detection model, and the integrity detection model to the compliance detection module. The compliance detection module collects real-time video footage of each worker. Based on the video and the wearing detection model, it determines whether the worker is wearing safety equipment. If not, a first warning is sent to the warning module. If the worker is wearing the equipment, based on the video and the correctness detection model, it determines whether the worker is wearing the safety equipment correctly. If not, a second warning is sent to the warning module. If the equipment is correctly worn, based on the video and the integrity detection model, it determines whether the worker is wearing the safety equipment completely. If complete, a safety wearing record is generated for the worker; if incomplete, a third warning is sent to the warning module. The early warning module generates a corresponding early warning record for the worker based on the early warning type, updates the worker's historical early warning set, and generates a fourth early warning for the worker based on the worker's historical early warning set.

Images