Method and system for identifying person standing under crane jib and medium

Abstract

The invention discloses a method and a system for identifying a person under a crane jib and a medium, and the method comprises the steps: collecting an image of the person under the crane jib, and carrying out the preprocessing to obtain an original data set; based on the original data set, constructing a detection model and performing training to obtain a first recognition model; after the original data set is enhanced, the first recognition model is trained again to obtain a second recognition model; and integrating the first identification model and the second identification model, and performing identification on the lower standing person of the crane jib by using the integrated model. According to the method, an external data set is introduced in combination with an actual problem to expand training samples and label, and the model precision can be improved through more sample training; in order to suppress overfitting of the model and improve model prediction robustness, data enhancement operation is introduced to train the first identification model again for fine tuning to obtain a second identification model, and the prediction results of the two models are fused and combined for post-processing to realize final prediction, so that the accuracy and robustness of prediction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power safety, and in particular to a method, system and medium for identifying a person standing under a crane boom. Background Art

[0002] The safety of power production is crucial to the stable operation of the power system. Among them, safety accidents caused by irregular operations in power production account for a considerable proportion, which seriously threatens the safety of production personnel and the normal operation of the power system. Therefore, it is of great significance to the safety of power production to accurately and quickly identify the risk of violations in on-site operations and improve the safety level of power on-site operations. In the actual production process, power operation cranes play an important role in power production. They can be mainly used for the installation and maintenance of power equipment, transportation and installation of transmission lines, rescue and repair, as well as handling and stacking. Cranes can provide stable lifting capacity and height for safe and efficient operation. However, there are also major safety hazards in the operation of cranes. One of the serious violations is people standing under the crane boom. When people stand under the crane boom, there are the following dangers: falling objects, objects on the crane boom may accidentally fall, causing injuries or even death; boom movement danger, the crane boom may suddenly move due to operating errors or other reasons, causing people to be clamped or fall; electric shock danger, the crane boom during power operations may contact high-voltage wires, there is a risk of electric shock, and people standing below are prone to electric shock. Therefore, it is strictly forbidden to stand under the crane boom to ensure the safety of personnel.

[0003] Based on the safety hazards existing in the above-mentioned crane operation, relevant supervision mechanisms have been gradually formed at the power operation site, but they are limited to arranging relevant supervisors near the crane operation site or traditional video surveillance to conduct on-site supervision. However, this supervision method has a high lag. No matter how the on-site patrol personnel or security personnel detect and intervene, there will always be blind spots and a certain time lag, and there are still great safety hazards. Summary of the invention

[0004] The present invention aims to overcome the problem that in the prior art, whether there is a person standing under the crane boom is supervised only by arranging supervisors to patrol manually on site or through traditional video surveillance, which has blind spots and time lags and still poses a great safety hazard. A method, system and medium for identifying people standing under the crane boom are provided.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A method for identifying a person standing under a crane boom, comprising: Collect images of people standing under the crane boom and perform preprocessing to obtain the original data set; Based on the original data set, a detection model is constructed and trained to obtain a first recognition model; After enhancing the original data set, the first recognition model is retrained to obtain a second recognition model; The first recognition model and the second recognition model are integrated, and the integrated model is used to recognize the person standing under the crane arm.

[0006] In view of the potential safety hazards of people standing under the crane arm during crane operation, the present invention proposes a target detection algorithm for identifying people under the crane arm based on Ensemble-YOLOv5s, so as to realize rapid identification of people under the crane arm and reduce the risk of work on the job site; firstly, an external data set is introduced to expand the training samples and annotate them in combination with actual problems, and more sample training can improve the accuracy of the model; then, the yolov5s algorithm is introduced to identify people under the crane arm, and the first recognition model is obtained through training. While ensuring the accuracy of reasoning, the yolov5s algorithm has faster reasoning speed and smaller calculation amount, and is easy to deploy on mobile terminals (control balls, smart terminals) and servers; on this basis, in order to suppress overfitting of the model and improve the robustness of model prediction, data enhancement operation is introduced to retrain the first recognition model for fine-tuning to obtain the second recognition model; finally, the prediction results of the two models are fused and integrated to realize the final prediction; the needs of high-precision and high-efficiency detection in target scenarios in the power system are met; the intelligent management and control capabilities of the power system are greatly improved to ensure the safety of people, power grids and equipment.

[0007] Preferably, the process of preprocessing to obtain the original data set includes: For the collected images of people standing under the crane boom, perform sample annotation in a unified form; Convert the labeled data of the sample into a data format that conforms to the input form of the detection model, including identifying the target category and target location parameters.

[0008] Preferably, the process of integrating the first recognition model and the second recognition model comprises: Inputting the image to be recognized into the first recognition model and the second recognition model respectively to obtain the corresponding first prediction result and the second prediction result; Post-processing the first prediction result and the second prediction result by using a non-maximum suppression algorithm; The first prediction result and the second prediction result are averaged to obtain an integrated fusion prediction result.

[0009] Preferably, the first prediction result and the second prediction result are both in the form of a set of bounding boxes containing the identified target and corresponding confidence levels; The first prediction result is post-processed to obtain a first bounding box containing the target position parameters; the second prediction result is post-processed to obtain a second bounding box containing the target position parameters; The first bounding box and the second bounding box are averaged to obtain the bounding box corresponding to the integrated prediction result.

[0010] Preferably, the non-maximum suppression algorithm comprises: Sort a set of bounding boxes from high to low according to confidence; traverse all bounding boxes starting from the bounding box with the highest confidence; for the current bounding box, if its overlap with the previously retained bounding box is greater than the set overlap threshold, suppress and exclude the bounding box; repeat the traversal process until all bounding boxes are processed.

[0011] Preferably, the detection model is constructed including: Input layer, input the image of a person standing under the crane arm; The backbone network includes a focusing structure for slicing the input image and a main network for extracting features from the input image; the neck network performs multi-scale feature fusion on the features extracted by the backbone network; Detection head, generates the final prediction results.

[0012] Preferably, the loss function of the detection model during training is obtained by weighted summation of classification loss, confidence loss and positioning loss; The classification loss evaluates the prediction effect of the model on the target category; the positioning loss evaluates the prediction effect of the model on the target location; and the confidence loss evaluates the credibility of the model in identifying the target.

[0013] Preferably, the enhancing process of the original data set comprises: Tone enhancement that adjusts the image hue; saturation enhancement that adjusts the image saturation; brightness enhancement that adjusts the image brightness; mixing enhancement that mixes two random images in proportion and distributes the mixed result in proportion.

[0014] A crane arm under the person identification system, comprising: The image processing module performs preprocessing and data enhancement on the collected images of people standing under the crane boom; A first model module is trained based on the original data set to obtain a first recognition model; The second model module is trained based on the original data set after data enhancement to obtain a second recognition model; The prediction output module integrates the first recognition model and the second recognition model to output the recognition result.

[0015] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying a person standing under a crane boom of the present application are implemented.

[0016] The present invention has the following beneficial effects: an Ensemble-YOLOv5s-based target detection algorithm for identifying people under the crane boom is proposed to realize rapid identification of people under the crane boom and reduce the risk of work on the job site; an external data set is introduced to expand the training samples and annotate them in combination with practical problems, and more sample training can improve the model accuracy; in order to suppress overfitting of the model and improve the robustness of model prediction, a data enhancement operation is introduced to re-train the first recognition model for fine-tuning to obtain a second recognition model, the prediction results of the two models are fused, and the final prediction is achieved in combination with post-processing, thereby improving the accuracy and robustness of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a flow chart of a method for identifying a person standing under a crane boom.

[0018] Figure 2 It is a schematic diagram of the mAP indicator and the loss function after the number of epochs of training is increased in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0020] like Figure 1 As shown, a method for identifying a person standing under a crane boom comprises: Collect images of people standing under the crane boom and perform preprocessing to obtain the original data set; Based on the original data set, a detection model is constructed and trained to obtain a first recognition model; After enhancing the original data set, the first recognition model is retrained to obtain a second recognition model; The first recognition model and the second recognition model are integrated, and the integrated model is used to recognize the person standing under the crane arm.

[0021] It should be noted that, in the present invention, a target detection algorithm for identifying people under the crane arm based on Ensemble-YOLOv5s is proposed to address the safety hazards of people standing under the crane arm during crane operation, so as to realize rapid identification of people under the crane arm and reduce the risk of work on the job site; firstly, an external data set is introduced to expand the training samples and annotate them in combination with actual problems, and more sample training can improve the accuracy of the model; then, the yolov5s algorithm is introduced to identify people under the crane arm, and the first recognition model is obtained through training. While ensuring the accuracy of reasoning, the yolov5s algorithm has a faster reasoning speed and a smaller amount of calculation, and is easy to deploy on mobile terminals (control balls, smart terminals) and servers; on this basis, in order to suppress overfitting of the model and improve the robustness of model prediction, a data enhancement operation is introduced to retrain the first recognition model for fine-tuning to obtain a second recognition model; finally, the prediction results of the two models are fused, and the final prediction is realized after integration; the needs of high-precision and high-efficiency detection in target scenarios in the power system are met; the intelligent management and control capabilities of the power system are greatly improved to ensure the safety of people, power grids, and equipment.

[0022] As a specific embodiment, the process of preprocessing to obtain the original data set includes: For the collected images of people standing under the crane boom, perform sample annotation in a unified form; Convert the labeled data of the sample into a data format that conforms to the input form of the detection model, including identifying the target category and target location parameters.

[0023] It should be noted that the data set for crane arm under-station identification mainly comes from three aspects, namely the training set images of the initial training set, the crane images crawled from the Internet, and the crane violation images collected by ourselves, totaling 472 training set images, and the test set to be predicted contains 100 images. The specific data is shown in the following table: Table 1 Sources of the crane arm under-arm human recognition dataset Serial number Data Source Number of samples Marking format 1 Initial training set 50 Marked 2 Web crawling 259 Self-labeling 3 Self-collected pictures of crane violations 163 Self-labeling In the specific data preprocessing process, since the initial training set samples are too small, it is difficult to train the model to obtain better detection results, so an external data set is introduced for self-labeling. Before labeling, first check the label image style of the initial training set, refer to the official labeling style, and introduce the labelimg tool to supplement the data set labeling.

[0024] After completing the unified form of sample annotation, the annotation data is converted. Since the annotation data refers to the official samples and is annotated in VOCXML format, refer to the input form of the yolo algorithm and write a function to implement the conversion of XML files to txt files. Specifically, the first column is the target category label, and the following four numbers are [x_center, y_center, w, h], where the label value of 0 represents the boom, the label value of 1 represents the hook, the label value of 2 represents the person under the boom, and the label value of 3 represents other people. [x_center, y_center, w, h] corresponds to the representative box coordinates of the target, which is the same as the bounding box output by the recognition model; x and y represent the center point coordinates of the box, and w and h represent the length and width of the box.

[0025] As a specific embodiment, the detection model is constructed including: The input layer inpuot inputs the image of a person standing under the crane arm.

[0026] Backbone network, including the focus structure for slicing the input image and the backbone network for feature extraction of the input image; Focus structure: Slicing the input image and then increasing the depth of the feature map through the convolution layer; The backbone network consists of multiple cross-stage architecture convolution blocks, which improve the efficiency of feature extraction by using cross-stage structures. YOLOv5s uses a smaller depth multiple parameter, which means that the depth of the network is shallower, thereby reducing the amount of calculation.

[0027] In Yolov5, common Backbone networks include CSPDarknet53 or ResNet. These networks are relatively lightweight and can minimize the amount of computation and memory usage while ensuring high detection accuracy. Its structure mainly includes Conv module, C3 module, and SPPF module. The Conv module is mainly composed of convolution layer, BN layer and activation function. The C3 module adaptively aggregates the previous feature map. The SPPF module obtains more comprehensive spatial information through weighted fusion of global features and local features.

[0028] The neck network performs multi-scale feature fusion on the features extracted by the backbone network. It mainly includes a feature pyramid network and a path aggregation network structure, which are used to further process feature maps and strengthen the fusion of features at different scales.

[0029] The Neck part is responsible for multi-scale feature fusion of the features extracted by Backbone and passing these features to the prediction layer. In the PANet structure adopted by Yolov5, the aggregation strategy is designed through multiple upsampling, splicing, points and dot products to better utilize multi-scale features.

[0030] Detection head, generates the final prediction results; includes multiple convolutional layers to generate the final detection results. The detection head structure of YOLOv5s is relatively simple because it is designed for a smaller model. The output layer predicts the bounding box, category probability, and confidence.

[0031] It should be noted that the detection model in this embodiment adopts the YoloV5s network. Yolov5 is an object detection architecture and model series pre-trained on the COCO public dataset. It is a derivative of the yolo series. Its network structure is divided into four modules: input input and output layer, backbone network, neck network and head detection head. The advantages of yolov5 over the previous yolo version network are: Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling are used at the input end; Focus structure and CSP structure are used at the backbone end; FPN+PAN structure is added at the neck end; the loss function during training is improved at the head end, using GIOU_Loss and DIOU_nms for bounding box screening. Yolov5s is an improved version based on the yolov5 model. yolov5s has the following advantages: high efficiency. Due to the use of some acceleration technologies based on new convolution modules, its biggest advantage is that compared with other versions of yolov5, while ensuring the accuracy of reasoning, the reasoning speed is faster and the amount of calculation is smaller. By comparison, yolov5s is easy to deploy on mobile terminals (mobile phones, smart terminals, and control balls) and servers. With high accuracy, yolov5s can combine feature maps of various sizes to generate a global feature description, so it has strong robustness for small objects, occlusion, lighting and other problems. In addition, yolov5s also uses a multi-scale training method, and performs appropriate processing for objects of different scales, which improves the accuracy of classification and positioning. The network structure is clear and the degree of modularization is high, which is convenient for adjusting and designing corresponding models.

[0032] Furthermore, during the training process of the detection model, the original data set is input into the detection model for training to obtain the first recognition model. During the training, the parameters of the model use the default parameters, specifically: the initial learning rate is set to 0.01, which is the learning rate at the beginning of the training; the final learning rate is set to 0.1, the final value of the single-cycle learning rate; the weight decay is set to 0.0005 to prevent overfitting; the stochastic gradient descent momentum is set to 0.937 to accelerate the convergence of the neural network optimizer; the learning rate warm-up is set to 3.0 to gradually increase the learning rate during the first three iterations of the initial training; the learning rate warm-up momentum is set to 0.8 to set a smaller momentum during the warm-up period; the target box loss gain is set to 0.05; the category loss gain is set to 0.3; the target loss gain is set to 0.7.

[0033] The loss function when the detection model is trained is obtained by weighted summation of classification loss, confidence loss and positioning loss; The classification loss evaluates the prediction effect of the model on the target category. It calculates whether the target in the bounding box is correctly classified with the actual target. The smaller the classification loss, the more accurate the classification. The cross entropy loss function can be used for calculation. The positioning loss evaluates the prediction effect of the model on the target position. It calculates the error between the bounding box and the actual representative box. The smaller the positioning loss, the more accurate the positioning. The mean square error loss function can be used for calculation. The confidence loss evaluates the credibility of the target identified by the model. The smaller the confidence loss, the higher the credibility of the target. The cross entropy loss function can also be used for calculation.

[0034] As a specific embodiment, the process of integrating the first recognition model and the second recognition model includes: inputting the image to be recognized into the first recognition model and the second recognition model respectively, and obtaining corresponding first prediction results and second prediction results; Post-processing the first prediction result and the second prediction result by using a non-maximum suppression algorithm; The first prediction result and the second prediction result are averaged to obtain an integrated fusion prediction result.

[0035] It should be noted that the Ensemble integration algorithm is a process of strengthening the robustness of reasoning, which can improve the robustness of model prediction. Specifically, multiple different models are created to predict the results, and the integrated model then summarizes the predictions of each basic model to obtain the final prediction of the test data. The motivation for using an integrated model is to reduce the generalization error of the prediction. When the basic models are diverse and independent, the prediction error of the model is generally reduced when the integrated method is used. Even if the integrated model has multiple basic models in the model (the average or maximum value of the predictions of multiple models is calculated or the bounding box is stacked and then screened), it is still run and executed as a single model (ultimately, the prediction is made by rounding a comprehensive model). Among them, the creation of different models can select algorithm models with different principles, or different training set samples can be selected to obtain different versions of the same model. In an embodiment, a data enhancement algorithm is introduced to iteratively update the first recognition model completed with training, and a second recognition model trained after enhancement is obtained. The second recognition model and the first recognition model are integrated to obtain the final prediction result.

[0036] Furthermore, the steps of obtaining the second recognition model and integrating the first recognition model with the second recognition model are as follows.

[0037] (1) Input the original data set and train the yolov5s model (during training, the model parameters are randomly initialized, that is, training starts from scratch, and the training time is relatively long). After the training is completed, an optimal model, namely the first recognition model (the parameters of this model are obtained), can be obtained, and the model parameters are saved and recorded as yolov5s_1.

[0038] (2) Enhancement processing of the original data set includes: Tone enhancement that adjusts the image hue; saturation enhancement that adjusts the image saturation; brightness enhancement that adjusts the image brightness; mixing enhancement that mixes two random images in proportion and distributes the mixed result in proportion.

[0039] Specifically, data augmentation of the original data set includes the following data augmentation operations: Hue enhancement, such as setting it to 0.015, is used to adjust the hue of the image. Saturation enhancement, such as setting it to 0.7, is used to adjust the saturation of the image. Brightness enhancement, such as setting it to 0.4, is used to adjust the brightness of the image. Image rotation degree, indicating that the image is rotated. Image translation, such as setting it to 0.1, indicates that the image can be translated in the horizontal and vertical directions. Image scaling ratio, such as setting it to 0.9, indicates that the image can be scaled. Image upside down flip, such as setting it to 0.5, indicates that there is a 50% probability of upside down flipping. Image left-right flip, such as setting it to 0.5, indicates that there is a 50% probability of left-right flipping. Image mosaic splicing, such as setting it to 1.0, indicates that mosaic splicing is always performed. Image MixUp mixed enhancement, such as setting it to 0.1, indicates that there is a 10% probability of mixed enhancement.

[0040] For the selection of data enhancement parameters, you can choose the medium data enhancement profile that comes with yolov5.

[0041] (3) After completing the data enhancement, the enhanced and expanded original data set is input into the detection model for training (during training, the model parameters are not randomly initialized, but the newly obtained model parameters yolov5s_1 are loaded into the detection model for fine-tuning. In this way, the model already has prior knowledge, and the yolov5s_1 model has learned rich feature representations, which can usually achieve better performance than training from scratch and accelerate the training process). The second recognition model yolov5s_2 is obtained after retraining.

[0042] (4) After obtaining the two models yolov5s_1 and yolov5s_2, perform model integration. The first prediction result and the second prediction result are both a set of bounding boxes containing the identified target and the corresponding confidence level; the first prediction result is post-processed to obtain a first bounding box containing the target position parameters; the second prediction result is post-processed to obtain a second bounding box containing the target position parameters; the first bounding box and the second bounding box are averaged to obtain the bounding box corresponding to the integrated prediction result. The model integration steps are as follows: 4.1) Independent Prediction: The yolov5s_1 and yolov5s_2 models independently make predictions on the input image, generating their own set of bounding boxes and corresponding confidence scores.

[0043] 4.2) Result merging: The prediction results of different models are concatenated at the feature level (along the channel dimension) to form a larger prediction tensor.

[0044] 4.3) Post-processing: The combined prediction results are processed together using the non-maximum suppression algorithm. The non-maximum suppression algorithm is used to remove overlapping detection boxes and only retain the boxes that are most likely to contain the target based on confidence and overlap.

[0045] 4.4) Integration decision: Adopt the average integration strategy to further improve the accuracy and robustness of the model. If the bounding box of yolov5s_1 is The bounding box of yolov5s_2 is Then the integrated frame is the average of the two, that is,

[0046] The non-maximum suppression algorithm includes: Sort a set of bounding boxes from high to low according to confidence; traverse all bounding boxes starting from the bounding box with the highest confidence; for the currently traversed bounding box, if its confidence is lower than the confidence threshold, suppress it; For the currently traversed bounding box, if its overlap with the previously retained bounding box is greater than the set overlap threshold, the bounding box is suppressed and excluded; the traversal process is repeated until all bounding boxes are processed.

[0047] A crane arm under the person identification system, comprising: The image processing module performs preprocessing and data enhancement on the collected images of people standing under the crane boom; A first model module is trained based on the original data set to obtain a first recognition model; The second model module is trained based on the original data set after data enhancement to obtain a second recognition model; The prediction output module integrates the first recognition model and the second recognition model to output the recognition result.

[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying a person standing under a crane boom of the present application are implemented.

[0049] In an embodiment of the present invention, for the improved method of identifying people standing under the crane boom, before performing model comparison, it is necessary to introduce a baseline model as a standard model for judgment, and perform algorithm tuning by comparing the improved model with the baseline model. In this embodiment, the baseline model directly selects 50 samples of the initial training set (one of which fails to read) as the training set for training, and the validation set also selects the data set of the 50 samples as the object for generating validation indicators. The training parameters epoch = 8, batchsize = 4, and imagesize = 640 are selected. The various validation set indicators are shown in Table 2 below: Table 2 Index data predicted by the baseline model Class Images Instances P R mAP All 49 198 0.817 0.209 0.264 Boom 49 57 0 0 0.0106 Hook 49 92 0.461 0.007 0.142 Man under the crane 49 44 0.805 0.75 0.9 Others 49 5 1 0 0.00381 In the table, P stands for precision, which indicates how many samples predicted as positive are correct; R stands for recall, which indicates how many of the truly positive samples are correctly predicted as positive; mAP stands for average precision. Figure 2 It can be seen that the performance of various indicators of the model is poor, and the mAP value of the overall data set is only 0.264. Although the mAP for the category of "person under the crane arm" reaches 0.9, the performance of other indicators is poor. Therefore, when expanding the data set, crane images are added to improve the recognition accuracy of the "hook" and "crane arm" categories.

[0050] The baseline model only uses native images without performing data enhancement operations and model fusion, so the recognition effect is poor. The final result obtained by the method proposed in this embodiment is compared with the recognition result of the baseline model, which reflects the superiority of the model of this embodiment.

[0051] Considering the performance of the baseline model, based on the sample data set of the 50 initial training sets, an external self-annotated data set is introduced for expansion, and the expanded data set is used for training. The training parameters of the baseline model are the same, and the 50 sample images of the original data set are used as the validation set to compare and verify the indicators. As shown in Table 3: Table 3 Index data for prediction after introducing self-standard samples Class Images Instances P R mAP All 49 198 0.584 0.824 0.697 Boom 49 57 0.883 0.421 0.713 Hook 49 92 0.557 0.967 0.94 Man under the crane 49 44 0.772 1 0.994 Others 49 5 0.124 0.907 0.139 After adding self-labeled samples, the indicators were significantly improved, and all indicators of the four categories were greatly improved. By comparison, it can be seen that after introducing external data sets, the indicators of the model were significantly improved. The mAP value of the overall data set reached 0.697, the mAP of the "hook" category increased to 0.94, and the mAP of the "arm" category increased to 0.713, both of which were doubled.

[0052] On the basis of the above, the model training epoch is increased (set to 64), and the input image size parameter is reduced, and the training is performed again. The various indicators of model verification are shown in Table 4 and Figure 2 As shown: Table 4 adds the final prediction indicator data after the training epoch Class Images Instances P R mAP All 49 198 0.919 0.984 0.995 Boom 49 57 1 0.95 0.995 Hook 49 92 0.978 0.987 0.994 Man under the crane 49 44 0.991 1 0.995 Others 49 5 0.707 1 0.995 After the training is finally completed, the mAP prediction index of the model for all categories of the validation set can reach 0.995; after about 30 epochs, the model indicators tend to converge, and after about 40 epochs, the loss function tends to converge and reaches the global optimum.

[0053] To prevent the model from overfitting on a small sample data set, an ensemble model is introduced for integration, sample transformation is achieved through data enhancement, and the model is retrained. Since the training is performed under the optimal model weight (the first recognition model), a large epoch (8 epochs are selected) is not required for iteration, and fine-tuning on the optimal weight is sufficient. The optimal model weight refers to the first recognition model obtained by training only with the expanded data set before data enhancement, and the model weight refers to the parameters of the model after model training. Fine-tuning refers to directly loading the optimal parameters obtained during model training, rather than randomly initialized parameters (randomly initialized parameters are all random numbers, which require multiple trainings and are slow). Here, the parameters just obtained are loaded because the model parameters already have recognition capabilities. Directly training the samples after data enhancement on this parameter will make the model converge faster and achieve better results.

[0054] The two models obtained from the above training are integrated and fused (frame stacking method), and the integrated fusion model is used to predict the validation set. The prediction indicators are shown in Table 5: Table 5 Index data for prediction by integrated fusion model Class Images Instances P R mAP All 49 198 0.96 0.952 0.995 Boom 49 57 1 0.832 0.995 Hook 49 92 0.989 0.978 0.994 Man under the crane 49 44 0.995 1 0.995 Others 49 5 0.856 1 0.995 Compared with the previous prediction index graph, the integrated fusion did not lead to a decrease in the model accuracy index, but the introduction of multiple models for fusion can improve the prediction robustness.

[0055] The crane arm person recognition algorithm based on Ensemble-YOLOv5s proposed in this embodiment has good recognition performance, and the verification set index mAP index can reach 0.995, which can accurately identify the situation of people standing under the crane arm. In terms of application prospects, the yolov5s model has fewer model parameters and faster reasoning response speed, and has been widely deployed on mobile devices such as mobile phones. Therefore, in actual power production, the algorithm can be extended to control balls, smart terminals, monitoring background servers and other equipment, occupying less resources, and the recognition accuracy is not lower than the large model version of the yolo series, which can promote the level of intelligent management and control of power production sites, reduce personal injuries and economic losses that may be caused during crane operations, and have obvious social and economic benefits.

[0056] The above embodiments are further elaborations and illustrations of the present invention for ease of understanding, and are not limitations of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying a person standing under a crane boom, characterized in that: include: Collect images of people standing under the crane boom and perform preprocessing to obtain the original data set; Based on the original data set, a detection model is constructed and trained to obtain a first recognition model; After enhancing the original data set, the first recognition model is retrained to obtain a second recognition model; The first recognition model and the second recognition model are integrated, and the integrated model is used to recognize the person standing under the crane arm.

2. A method for identifying a person standing under a crane boom according to claim 1, characterized in that: The process of preprocessing to obtain the original data set includes: For the collected images of people standing under the crane boom, perform sample annotation in a unified form; Convert the labeled data of the sample into a data format that conforms to the input form of the detection model, including identifying the target category and target location parameters.

3. A method for identifying a person standing under a crane boom according to claim 1 or 2, characterized in that: The process of integrating the first recognition model and the second recognition model comprises: Inputting the image to be recognized into the first recognition model and the second recognition model respectively to obtain the corresponding first prediction result and the second prediction result; Post-processing the first prediction result and the second prediction result by using a non-maximum suppression algorithm; The first prediction result and the second prediction result are averaged to obtain an integrated fused prediction result.

4. A method for identifying a person standing under a crane boom according to claim 3, characterized in that: The first prediction result and the second prediction result are both a set of bounding boxes containing the identified object and corresponding confidence levels; The first prediction result is post-processed to obtain a first bounding box containing the target position parameters; the second prediction result is post-processed to obtain a second bounding box containing the target position parameters; The first bounding box and the second bounding box are averaged to obtain the bounding box corresponding to the integrated prediction result.

5. A method for identifying a person standing under a crane boom according to claim 4, characterized in that: The non-maximum suppression algorithm includes: Sort a set of bounding boxes by confidence from high to low; traverse all bounding boxes starting from the bounding box with the highest confidence; For the current bounding box, if its overlap with the previously retained bounding box is greater than the set overlap threshold, the bounding box is suppressed and excluded; the traversal process is repeated until all bounding boxes are processed.

6. A method for identifying a person standing under a crane boom according to claim 3, characterized in that: The detection model includes: Input layer, input the image of a person standing under the crane arm; A backbone network, including a focusing structure for slicing an input image and a backbone network for extracting features from the input image; The neck network performs multi-scale feature fusion on the features extracted by the backbone network; Detection head, generates the final prediction results.

7. A method for identifying a person standing under a crane boom according to claim 6, characterized in that: The loss function of the detection model during training is obtained by weighted summation of classification loss, confidence loss and positioning loss; The classification loss evaluates the prediction effect of the model on the target category; the positioning loss evaluates the prediction effect of the model on the target location; and the confidence loss evaluates the credibility of the model in identifying the target.

8. A method for identifying a person standing under a crane boom according to claim 1, 2, 4, 5, 6 or 7, characterized in that: The enhancement processing of the original data set includes: Tone enhancement that adjusts the image hue; saturation enhancement that adjusts the image saturation; brightness enhancement that adjusts the image brightness; mixing enhancement that mixes two random images in proportion and distributes the mixed result in proportion.

9. A system for identifying people standing under a crane boom, applicable to the method for identifying people standing under a crane boom as claimed in any one of claims 1 to 8, characterized in that: include: The image processing module performs preprocessing and data enhancement on the collected images of people standing under the crane boom; A first model module is trained based on the original data set to obtain a first recognition model; The second model module is trained based on the original data set after data enhancement to obtain a second recognition model; The prediction output module integrates the first recognition model and the second recognition model to output the recognition result.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying a person standing under a crane boom as claimed in any one of claims 1 to 8 are implemented.

Images