Labor protection article target detection method and system based on incremental learning

Abstract

The present application belongs to the technical field of computer vision, and particularly relates to a kind of safety articles target detection method and system based on incremental learning, method includes constructing first safety articles target detection dataset to train teacher network;Second safety articles target detection dataset is obtained, according to the label category number more than first safety articles target detection dataset in second safety articles target detection dataset, increase the number of student network output layer neurons to expand network;Student model initialization is carried out, and distillation loss function is constructed between student model and teacher model;Loss function of student model is constructed based on the constructed distillation loss function, and student model is trained using second safety articles target detection dataset;The trained student model is deployed to wisdom construction site platform, and target detection is carried out in construction site;The present application overcomes the problem that traditional deep learning algorithm will produce catastrophic forgetting after training on new sample.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and specifically relates to a method and system for detecting personal protective equipment targets based on incremental learning. Background Technology

[0002] Wearing personal protective equipment (PPE) is the most basic measure to protect personnel safety in infrastructure construction projects. The detection mechanism for whether PPE is worn forms a safety barrier for workers, and improving the practicality of target detection algorithms is of great significance to intelligent construction site systems. Similarly, conducting research on PPE target detection is key to improving safety supervision capabilities. In recent years, thanks to the continuous development of deep learning theory and the emergence of large-scale target picking datasets, research on target detection has made groundbreaking progress.

[0003] When researching object detection algorithms, researchers typically assume that the training and test sets are independent and identically distributed, meaning all image data are collected from the same distribution and are mutually independent. This assumption usually holds true when the training and test sets come from the same dataset. However, due to the specific nature of construction scenarios, data from actual construction sites is often scarce, and these scenarios continuously change as construction progresses. Therefore, in the data collection process for actual construction site operations, data is often added in multiple batches of small quantities. When the target type changes, targets not present in the training set cannot be accurately identified.

[0004] When the target type changes, the model needs to be retrained. Training with only new data samples without incremental learning can lead to catastrophic forgetting of existing knowledge. Retraining a model from scratch every time new data is collected involves enormous computational overhead, lacks scalability, and is inefficient with data. Incremental learning, on the other hand, allows training with new data on top of the existing model, preventing the model from forgetting previous knowledge. This continuous learning capability is crucial for automated detection systems to handle changes in the required personal protective equipment (PPE) targets during actual construction. Summary of the Invention

[0005] To adapt to changes in the type of personal protective equipment (PPE) during PPE inspection, this invention proposes a PPE target detection method based on incremental learning, which specifically includes the following steps:

[0006] Images of safety protection equipment targets in a real construction environment are captured by construction site cameras. The captured images are then labeled with category and location information to form the first safety protection equipment target detection dataset.

[0007] An object detection network was constructed as a teacher model, and the teacher network was trained using the first set of labor protection item object detection datasets.

[0008] The second labor protection equipment target detection dataset is formed by capturing images of labor protection equipment targets in a real construction environment using construction site cameras, and only labeling the new categories of labor protection equipment that have not appeared in the first labor protection equipment target detection dataset with category and location information.

[0009] Based on the number of label categories in the second labor protection item target detection dataset, the number of neurons in the student network output layer is increased to expand the network;

[0010] The student network is initialized using the network parameters of the trained classroom network, and newly added neurons in the student model are randomly initialized.

[0011] Construct a distillation loss function between the student model and the teacher model;

[0012] The loss function of the student model is constructed based on the distillation loss function, and the student model is trained using the second labor protection item target detection dataset.

[0013] The trained student model is deployed on the smart construction site platform to perform target detection within the construction site.

[0014] Furthermore, a distillation loss function is constructed between the student model and the teacher model. The distillation loss function includes:

[0015] The loss function between the last layer outputs of the student model and the teacher model;

[0016] The loss function between the outputs of the last layer of the backbone feature extraction networks for the student and teacher models;

[0017] The loss function between the classification branch outputs of the teacher network and the student network for the old category;

[0018] The loss function between the regression branch outputs of the teacher network and the student network for the old category.

[0019] Furthermore, the student model was trained using the second dataset for detecting targets in occupational safety equipment. The loss function used during training was:

[0020]

[0021]

[0022] Among them, L dist_total This represents the loss function used to train the student model using the second dataset of labor protection items target detection; Let F represent the loss function between the student model and the output of the last layer of the backbone feature extraction network of the teacher model, and let F be the feature map of the last layer of the backbone feature extraction network of the teacher model. Extract feature maps of the last layer of the student network backbone; Let P be the loss function between the outputs of the last layer of the student model and the teacher model, and let P be the feature map of the last layer of the teacher network model. The feature map of the last layer of the student network model; C represents the loss function between the teacher network and the student network for the classification branch output of the old category. T For the teacher network's classification branch output for the old category, C S Output the classification branches for the old categories in the student network; Let B be the loss function between the regression branch outputs of the teacher network and the student network for the old category. T B is the probability matrix for each bounding box predicted by the teacher network model. S λ1 and λ2 are the probability matrices for each bounding box predicted by the student network model; λ1 and λ2 are balance factors.

[0023] Furthermore, the loss function between the last layer output of the backbone feature extraction network of the student model and the teacher model... Represented as:

[0024]

[0025] Among them, F i This represents the output of the last layer feature map of the backbone feature extraction network for the i-th input sample. Let represent the output of the last layer feature map of the student network backbone feature extraction network for the i-th input sample; m is the number of samples processed simultaneously in a single training session.

[0026] Furthermore, the loss function between the student model and the teacher model's final layer output... Represented as:

[0027]

[0028] Among them, P i For the final output layer of the teacher network, is the output of the last layer of the student network, and m is the number of samples processed simultaneously in a single training iteration.

[0029] Furthermore, the loss function between the teacher network and the student network for the classification branch output of the old category... Represented as:

[0030]

[0031] in, The output of the teacher network is the classification branch of the old category for the i-th input sample. is the classification branch output of the student network for the old category of the i-th input sample, and m is the number of samples processed simultaneously in a single training session.

[0032] Furthermore, the loss function between the regression branch outputs of the teacher network and the student network for the old category... Represented as:

[0033]

[0034]

[0035]

[0036] Where J is the number of bounding boxes predicted by the model for the current sample; This indicates the calculation of the KL divergence of one of the boundaries of two bounding boxes; Let be the probability matrix of the j-th bounding box in the teacher network. Let p be the probability matrix of the j-th bounding box in the student network; t p b p l p r Let represent the probability distributions of the top, bottom, left, and right edges of the bounding boxes predicted by the model, respectively, and B be the probability matrix of each bounding box.

[0037] Furthermore, the loss function for training the teacher network using the occupational safety equipment target detection dataset is expressed as:

[0038]

[0039] Wherein, Loss is the loss function used when training the teacher network using the first labor protection item target detection dataset; Y represents the bounding box loss, and Y is the output of the teacher network for the classification category. For the classification of true values; B represents the bounding box loss, and B represents the bounding box location information output by the teacher network. The actual value of the bounding box; Here, C represents the confidence loss, and C is the probability distribution output by the teacher model, indicating the confidence information of the prediction result. This represents the true confidence level.

[0040] This invention also proposes a safety equipment target detection system based on incremental learning, which is used to implement a safety equipment target detection method based on incremental learning. The system includes a target detection server and cameras deployed at the construction site. The cameras collect images of the construction site and upload them to the server. The server uses a trained student model to perform target detection on the images.

[0041] This invention uses knowledge distillation to incrementally learn the targets of labor protection items, adapting to changes in the construction environment. This enables the target detection model to not only acquire the ability to recognize new targets but also retain the memory of the initial targets. It overcomes the problem of catastrophic forgetting that occurs after traditional deep learning algorithms are trained on new samples, and constructs a target detector that can learn continuously. Attached Figure Description

[0042] Figure 1 This is a flowchart of a method for detecting personal protective equipment targets based on incremental learning according to the present invention;

[0043] Figure 2 This is a schematic diagram of the incremental target detection network structure of the present invention;

[0044] Figure 3 This is a schematic diagram of the model of the present invention deployed on a smart construction site platform. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] This invention proposes a method for detecting personal protective equipment targets based on incremental learning, which specifically includes the following steps:

[0047] Images of safety protection equipment targets in a real construction environment are captured by construction site cameras. The captured images are then labeled with category and location information to form the first safety protection equipment target detection dataset.

[0048] An object detection network was constructed as a teacher model, and the teacher network was trained using the first set of labor protection item object detection datasets.

[0049] The second labor protection equipment target detection dataset is formed by capturing images of labor protection equipment targets in a real construction environment using construction site cameras, and only labeling the new categories of labor protection equipment that have not appeared in the first labor protection equipment target detection dataset with category and location information.

[0050] Based on the number of label categories in the second labor protection item target detection dataset, the number of neurons in the student network output layer is increased to expand the network;

[0051] The student network is initialized using the network parameters of the trained classroom network, and newly added neurons in the student model are randomly initialized.

[0052] Construct a distillation loss function between the student model and the teacher model;

[0053] The loss function of the student model is constructed based on the distillation loss function, and the student model is trained using the second labor protection item target detection dataset.

[0054] The trained student model is deployed on the smart construction site platform to perform target detection within the construction site.

[0055] In this embodiment, as Figure 1 The diagram shows a flowchart of a method for detecting personal protective equipment targets based on incremental learning, which includes the following steps:

[0056] (1) Construct an initial category dataset by capturing images of safety protection items in a real construction environment using construction site cameras, and labeling the initial safety protection items with category and location information to construct a safety protection item target detection dataset;

[0057] (2) Construct the teacher network model. The teacher network model uses an arbitrary object detection network as the teacher network. Read the initial dataset of labor protection items as the initial category, and train and test the model. The loss function is as follows:

[0058]

[0059] in, For bounding box loss, For classifying losses, For confidence loss

[0060] Once the loss value converges, the model is tested. If the loss value does not converge, the model hyperparameters are adjusted until the model converges.

[0061] (3) Construct a new category dataset: capture images of safety protection items in a real construction environment using construction site cameras, and label the new category of safety protection items with category and location information in the dataset to construct a safety protection item target detection dataset;

[0062] New categories refer to the target categories of labor protection items that did not appear in the initial category dataset, i.e., categories that were not input into the target detection network model.

[0063] (4) Construct a student network model. The overall structure of the student network is the same as that of the teacher network. Add corresponding classification subnetworks according to the number of new category targets. Assuming that the newly added labor protection item target category is n, n neurons need to be added to the output layer of the student network to expand the classification subnetwork. The parameters in the student network are initialized using the parameters of the trained teacher network, and the newly added neurons in the classification subnetwork are randomly initialized.

[0064] (5) Construct a distillation loss function between the teacher network and the student network to ensure that the student network can fully learn the knowledge of the teacher network. The distillation loss function mainly consists of four parts. First, a loss function is established between the last layer of the entire network, as shown in the following formula:

[0065]

[0066] Where P represents the feature map of the last layer of the teacher network model. This represents the feature map of the last layer of the student network model.

[0067] The second part establishes a construction loss function between the last layer of the backbone feature extraction networks of the teacher network and the student network, as shown in the following formula:

[0068]

[0069] Where F represents the feature map of the last layer of the teacher network backbone feature extraction network. The feature map of the last layer of the network is extracted from the backbone features of the student network.

[0070] The third part constructs a loss function between the classification branch outputs of the teacher network and the student network for the old category, as shown in the following formula:

[0071]

[0072] Where C T C represents the output of the classification branch of the teacher network for the old category. S This represents the output of the classification branch of the student network for the old category.

[0073] Part Four constructs a loss function between the regression branch outputs of the teacher network and the student network for the old category, as shown in the following formula:

[0074]

[0075]

[0076]

[0077] Where, p t pb p l p r Let L represent the probability distribution of each edge e of the bounding box predicted by the model, B be the probability matrix of each bounding box, and L be the probability distribution of each edge e. KL Let KL divergence be a metric.

[0078] (6) To train the incremental target detection network on the new category of labor protection products target dataset, the distillation loss and the loss during normal training of the target detection network need to be added together to obtain a new loss function, as shown in the following formula:

[0079]

[0080]

[0081] Here, λ1 and λ2 are used to balance the hyperparameters between different losses. By default, we use λ1 = λ2 = 1;

[0082] During training, images are simultaneously input into both the teacher network and the student network. The parameters of the teacher network are frozen, and only the parameters in the student network are updated until the loss value converges to the minimum.

[0083] (7) Deploy the trained model on the smart construction site platform to detect the target. Deploying the trained model on the smart construction site platform only requires deploying the student network on the smart construction site platform to complete the detection of new categories of labor protection items.

[0084] (8) When adding a new category of detection target, repeat steps 103 to 107.

[0085] like Figure 2 The diagram shows the structure of an incremental object detection network. In the diagram, A represents the teacher network model trained on the initial category dataset, and B represents the student network model that needs to incrementally learn new object categories. A distillation loss function is established between the student and teacher networks to learn the knowledge from the teacher network. The distillation loss function mainly consists of two parts: first, a loss function L is established between the last layer of the entire network. dist_output Secondly, a loss function L is established between the last layer of the backbone feature extraction networks of the teacher network and the student network. dist_feature . This represents the loss function of the student network when learning a new target category. During training, images are simultaneously input into both the teacher and student networks. The parameters of the teacher network are frozen, and only the parameters of the student network are updated until the loss value converges to its minimum.

[0086] In this embodiment, the neural network used for target detection is divided into two parts: a backbone extraction network for extracting features from data, and a target detection head for target detection based on the features extracted by the backbone extraction network, which is equivalent to a classification network. Those skilled in the art can choose any feature extraction unit and target detection unit in the prior art to construct the teacher model and student model. This embodiment does not limit the specific structure of the teacher model and student model.

[0087] like Figure 3 The diagram illustrates the deployment of the model on the smart construction site platform. When deploying the target detection model, simply porting the well-learned student network model to the embedded platform of the detection system is sufficient to detect safety equipment targets. During system operation, images of the construction site are obtained using surveillance cameras, input into the target detection network to obtain detection results, and finally, the results are input into the control system of the smart construction site platform, providing environmental information for the safety detection system.

[0088] In summary, the incremental learning-based method for detecting work safety items of this invention employs knowledge distillation to solve the catastrophic forgetting problem when learning new target categories. When the type of work safety item changes, incremental learning can be performed on the target, gaining the ability to recognize new targets while retaining the memory of the original targets. This constructs a continuously learning target detector, enabling the detection system to quickly acquire the ability to identify new categories of work safety items.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting personal protective equipment targets based on incremental learning, characterized in that, When the type of occupational safety equipment targets changes, incremental learning of occupational safety equipment targets can be performed, specifically including the following steps: Images of safety protection equipment targets in a real construction environment are captured by construction site cameras. The captured images are then labeled with category and location information to form the first safety protection equipment target detection dataset. An object detection network was constructed as a teacher model, and the teacher network was trained using the first set of labor protection item object detection datasets. The second labor protection equipment target detection dataset is formed by capturing images of labor protection equipment targets in a real construction environment using construction site cameras, and only labeling the new categories of labor protection equipment that have not appeared in the first labor protection equipment target detection dataset with category and location information. Based on the number of label categories in the second labor protection item target detection dataset, the number of neurons in the student network output layer is increased to expand the network; The student network is initialized using the network parameters of the trained classroom network, and newly added neurons in the student model are randomly initialized. A distillation loss function is constructed between the student model and the teacher model. This loss function includes the loss function between the classification branch outputs of the teacher network and the student network for the old category. , represented as: ； in, For the teacher network to the first The classification branch output for the old category of each input sample. The output of the classification branch for the old class of the student network for the nth input sample is... This represents the number of samples processed simultaneously in a single training session. The loss function between the regression branch outputs of the teacher network and the student network for the old category Represented as: ； ； ； in, This represents the number of bounding boxes predicted by the model for the current sample. This indicates the calculation of the KL divergence of one of the boundaries of two bounding boxes; For Teachers' Network The probability matrix of each bounding box. For student network The probability matrix of each bounding box; These represent the probability distributions of the top, bottom, left, and right edges of the bounding box predicted by the model, respectively. The probability matrix for each bounding box; The loss function of the student model is constructed based on the distillation loss function, and the student model is trained using the second labor protection item target detection dataset. The trained student model is deployed on the smart construction site platform to perform target detection within the construction site.

2. The method for detecting safety protection items based on incremental learning according to claim 1, characterized in that, A distillation loss function is constructed between the student model and the teacher model. The distillation loss function includes: The loss function between the last layer outputs of the student model and the teacher model; The loss function between the outputs of the last layer of the backbone feature extraction networks for the student and teacher models; The loss function between the classification branch outputs of the teacher network and the student network for the old category; The loss function between the regression branch outputs of the teacher network and the student network for the old category.

3. A method for detecting safety protection items based on incremental learning according to claim 1 or 2, characterized in that, The student model was trained using the second dataset for detecting targets in labor protection equipment. The loss function used during training was: ； ； in, This represents the loss function used to train the student model using the second dataset of labor protection items target detection; Let F represent the loss function between the student model and the output of the last layer of the backbone feature extraction network of the teacher model, and let F be the feature map of the last layer of the backbone feature extraction network of the teacher model. Extract feature maps of the last layer of the student network backbone; The loss function between the outputs of the last layer of the student model and the teacher model. This is the feature map of the last layer of the teacher network model. The feature map of the last layer of the student network model; This represents the loss function between the teacher network and the student network for the classification branch output of the old category. For the teacher network, the classification branch output for the old category is... Output the classification branches for the old categories in the student network; Let be the loss function between the regression branch outputs of the teacher network and the student network for the old category. This is the probability matrix for each bounding box predicted by the teacher network model. The probability matrix for each bounding box predicted by the student network model; , It is a balancing factor.

4. The method for detecting safety protection items based on incremental learning according to claim 3, characterized in that, The loss function between the last layer output of the backbone feature extraction networks of the student model and the teacher model Represented as: ； in, For the teacher network backbone feature extraction network, the first The output of the last layer feature map of each input sample To represent the student network backbone feature extraction network for the first The output of the last layer feature map of each input sample; This represents the number of samples processed simultaneously in a single training session.

5. The method for detecting safety protection items based on incremental learning according to claim 3, characterized in that, The loss function between the student model and the teacher model's last layer output Represented as: ； in, For the final output layer of the teacher network, For the last layer output of the student network, This represents the number of samples processed simultaneously in a single training session.

6. The method for detecting safety protection items based on incremental learning according to claim 1, characterized in that, The loss function for training the teacher network using a dataset of labor protection equipment target detection is expressed as: ； in, The loss function used when training the teacher network using the first dataset of labor protection items target detection; For bounding box loss, For the teacher network's output of classification categories, For the classification of true values; For bounding box loss, The bounding box location information output by the teacher network. The actual value of the bounding box; For confidence loss, The probability distribution output by the teacher model represents the confidence information of the prediction results. This represents the true confidence level.

7. A system for detecting personal protective equipment (PPE) targets based on incremental learning, characterized in that, To implement the incremental learning-based target detection method for labor protection items as described in claim 1, the system includes a target detection server and cameras deployed at the construction site. The cameras collect images of the construction site and upload them to the server. The server uses a trained student model to perform target detection on the images.

Images