Method for small human head detection based on attention mechanism and related device

Abstract

The present disclosure provides a small size head detection method based on attention mechanism and related equipment, and relates to the field of computer image. The method comprises: inputting a target image into a pre-trained first neural network to output a heat map, wherein the heat map reflects the position of the head; inputting the target image into a second neural network to output a feature map; fusing the heat map and the feature map to determine a weighted feature map; inputting the weighted feature map into a pre-trained third neural network to output a plurality of detection boxes; and determining the detection box with the maximum coordinate score in the plurality of detection boxes as the detection result according to the coordinates with a score greater than a preset threshold in the heat map. The present disclosure can improve the small size head detection accuracy by predicting the position of the head through the heat map of the first neural network and realizing the attention mechanism by fusing the heat map with the feature map of the second neural network.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer images, and particularly relates to a small-size human head detection method based on an attention mechanism and related equipment. BACKGROUND

[0002] At present, an automatic monitoring system based on video images and target detection algorithms has been widely applied to multiple fields in life. In a target detection task, a large number of picture data of real scenes need to be collected on site or searched on the network, and the collected picture data is manually labeled, then the collected data is input into a neural network, the neural network is trained for a task to be completed, and finally the picture to be predicted is input into the trained network model, and the output of the model is operated in a series of operations to analyze the model parameters to form a visual detection frame result.

[0003] In the prior art, in some special scenes such as a small target such as a construction site safety helmet scene, the commonly used detection algorithm cannot accurately recognize the small-size target.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The present disclosure provides a small-size human head detection method based on an attention mechanism and related equipment, which at least partially overcomes the problem of inaccurate small-size human head detection in the related art.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a small-size human head detection method based on an attention mechanism is provided, comprising: inputting a target image into a first neural network trained in advance to output a heat map, wherein the heat map reflects the position of a human head; inputting the target image into a second neural network to output a feature map; fusing the heat map and the feature map to determine a weighted feature map; inputting the weighted feature map into a third neural network trained in advance to output a plurality of detection frames, wherein the center point of the detection frame is in the center of the human head region; and determining, according to the coordinates with a score greater than a preset threshold in the heat map, a detection frame with the maximum coordinate score in the plurality of detection frames as a detection result.

[0008] In some embodiments, the pre-trained first neural network comprises: obtaining a target image; performing rectangular box labeling on a human head in the target image to determine a labeled box, wherein the labeled box comprises coordinate information and a center of the labeled box corresponds to a center of a human head region; determining a heat map according to an output of the first neural network after the target image is input into the first neural network; determining a ground truth heat map according to a center point of the rectangular box labeling; and training the first neural network according to the heat map and the ground truth heat map.

[0009] In some embodiments, the inputting the target image into the second neural network to output a feature map comprises: inputting the target image into the second neural network to output a feature map with a same size as the heat map.

[0010] In some embodiments, the pre-trained third neural network comprises: performing feature fusion on the feature map and the heat map by weighting to determine a weighted feature map; and training the third neural network according to the weighted feature map.

[0011] In some embodiments, the performing feature fusion on the feature map and the heat map by weighting to determine a weighted feature map comprises: multiplying the heat map and the feature map on each channel point by point to weight the feature map on each channel to determine a weighted feature map, wherein the heat map comprises one channel and the feature map comprises at least one channel.

[0012] In some embodiments, the method further comprises: updating parameters of the second neural network and the third neural network according to a back propagation algorithm.

[0013] In some embodiments, the method further comprises: deploying the first neural network in a first thread; and deploying the second neural network and the third neural network in a second thread.

[0014] According to another aspect of the present disclosure, a small size human head detection device based on an attention mechanism is also provided, comprising: a first neural network module configured to input a target image into a pre-trained first neural network to output a heat map, wherein the heat map reflects a position of a human head; a second neural network module configured to input the target image into a second neural network to output a feature map; a feature fusion module configured to perform feature fusion on the heat map and the feature map to determine a weighted feature map; a third neural network module configured to input the weighted feature map into a pre-trained third neural network to output a plurality of detection boxes; and a detection result determination module configured to determine a detection box with a maximum coordinate score in the plurality of detection boxes as a detection result according to a coordinate with a score greater than a preset threshold in the heat map.

[0015] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the attention-based small head detection method described above by executing the executable instructions.

[0016] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the attention-based small-size head detection method described in any one of the preceding claims.

[0017] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the attention-based small-sized head detection method described above.

[0018] The small-sized head detection method and related device based on an attention mechanism provided in the embodiments of this disclosure involve inputting a target image into a pre-trained first neural network, which outputs a heatmap reflecting the location of the head; inputting the target image into a second neural network, which outputs a feature map; fusing the heatmap and the feature map to determine a weighted feature map; inputting the weighted feature map into a pre-trained third neural network, which outputs multiple detection boxes; and determining the detection box with the highest coordinate score among the multiple detection boxes as the detection result based on the coordinates in the heatmap that have a score greater than a preset threshold. In the embodiments of this disclosure, because the heatmap of the first neural network predicts the location of the head and is fused with the feature map of the second neural network to achieve an attention mechanism, a more accurate head bounding box can be obtained, thereby contributing to the improvement of the accuracy of small-sized head detection.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 This diagram illustrates the structure of a small-sized head detection system based on an attention mechanism, according to an embodiment of the present disclosure.

[0022] Figure 2 This diagram illustrates another small-sized head detection method based on an attention mechanism in an embodiment of this disclosure.

[0023] Figure 3 This diagram illustrates a flowchart of another small-sized head detection method based on an attention mechanism in an embodiment of this disclosure.

[0024] Figure 4 This diagram illustrates a flowchart of another small-sized head detection method based on an attention mechanism, as shown in this embodiment of the present disclosure.

[0025] Figure 5 This diagram illustrates another small-sized head detection method based on an attention mechanism in an embodiment of this disclosure.

[0026] Figure 6 This diagram illustrates the overall training of a small-sized head detection method based on an attention mechanism according to an embodiment of the present disclosure.

[0027] Figure 7 This diagram illustrates the auxiliary network training of an attention-based small-sized head detection method according to an embodiment of the present disclosure.

[0028] Figure 8 This diagram illustrates a small-sized head detection device based on an attention mechanism according to an embodiment of the present disclosure.

[0029] Figure 9 This diagram illustrates a structural block diagram of a computer device according to an embodiment of the present disclosure;

[0030] Figure 10 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0034] Figure 1 A schematic diagram of an exemplary application system architecture is shown, illustrating an example of an attention-based small-sized head detection method applicable to embodiments of this disclosure. For example... Figure 1 As shown, the system architecture 100 may include an auxiliary network 101, a backbone network 102, and a detection network 103.

[0035] The auxiliary network (first neural network) is deployed on thread one and consists of multiple convolutional and pooling layers. It ultimately generates a heatmap H of the head position. The heatmap size (equivalent to the above dimensions) is w*h and the number of channels is 1. During training, the cross-entropy loss between the heatmap H generated by the auxiliary network and the ground truth heatmap H_GT is calculated point by point to train the auxiliary network.

[0036] After the auxiliary network is trained, the backbone network (second neural network) is trained. The backbone network also consists of multiple convolutional and pooling layers, which are responsible for extracting image features and finally extracting a feature map F. The size of the feature map is consistent with that of the heatmap, which is w*h, and the number of channels is C. During feature fusion, the heatmap and the feature map on each channel are multiplied point by point to generate a weighted feature map F1, which has a size of w*h and the number of channels is C.

[0037] Finally, the weighted feature map F1 is input into the detection network (third neural network) to train the detection network. During backpropagation, only the parameters of the detection network and the backbone network are updated. After obtaining a large number of candidate boxes through the detection network, the coordinates of all points with scores greater than the threshold T are taken in the heatmap generated by the auxiliary network. Among the many candidate boxes, the detection box with the largest score at its coordinate position and its four neighbors is selected as the final detection result. This replaces the non-maximum suppression (NMS) process of the traditional detection algorithm, thus speeding up the post-processing.

[0038] Before the system runs, the weight files of the auxiliary network, backbone network, and detection network are first converted to the format required by the corresponding chip manufacturer. The parameter precision can be appropriately reduced to improve the inference speed. The algorithm is deployed using two threads: the auxiliary network is deployed on thread 1 (first thread), and the backbone network and detection network are deployed on thread 2 (second thread). After thread 1 finishes running, the auxiliary network shares the heat map and the coordinates of the points with responses greater than the threshold with thread 2. After the backbone network finishes its calculation, it will use the information shared by thread 2 for subsequent calculations.

[0039] This disclosure makes full use of the characteristics of objects such as human heads and safety helmets, which have relatively fixed shapes and small scale changes under the same monitoring scene. It uses an additional auxiliary network to predict the position of the center of the human head in the image, and uses the position of the center of the human head to assist the training of the backbone network and the detection network. In the post-processing process, the final detection box is obtained directly from the candidate box.

[0040] This disclosure can significantly improve the accuracy and recall of small targets in images, and eliminates the need for time-consuming non-maximum suppression in post-processing, thereby improving algorithm speed.

[0041] The auxiliary network module disclosed herein is decoupled from the backbone network and detection network module during the inference process. During deployment, the two modules can be placed in different processes or threads for parallel processing, and can run offline in real time on embedded devices from any vendor.

[0042] Those skilled in the art will know that Figure 1 The number of auxiliary networks, backbone networks, and detection networks shown is merely illustrative; any number of auxiliary networks, backbone networks, and detection networks can be used as needed. This disclosure does not limit this.

[0043] Under the above system architecture, this disclosure provides a small-sized head detection method based on an attention mechanism, which can be executed by any electronic device with computing power.

[0044] Figure 2 This invention discloses a flowchart of a small-sized head detection method based on an attention mechanism, as shown in the embodiments of the present invention. Figure 2 As shown, the small-sized head detection method based on the attention mechanism provided in this embodiment includes the following steps:

[0045] S202, the target image is input into the pre-trained first neural network, and a heat map is output, in which the heat map reflects the location of the human head.

[0046] It should be noted that the target image mentioned above can be an image of a person wearing a safety helmet, captured in a head detection scenario. For example, an image captured in a construction site safety helmet scenario. The first neural network model mentioned above can be a neural network model, which is a mathematical model described based on the mathematical model of neurons. The heatmap mentioned above can be an illustration that displays the page areas favored by visitors and the geographical area where visitors are located in a specially highlighted form. For example, highlighting an image of a person wearing a safety helmet.

[0047] S204: Input the target image into the second neural network and output a feature map.

[0048] It should be noted that the second neural network model mentioned above can be a neural network model. The feature map mentioned above can be a two-dimensional image. For example, in each convolutional layer, the data exists in a three-dimensional form, which is many two-dimensional images stacked together, each of which is called a feature map.

[0049] S206, fuse the heatmap and feature map features to determine the weighted feature map.

[0050] It should be noted that the above feature fusion can be an optimized combination of different feature vectors extracted from the same pattern. For example, during feature fusion, the heatmap and the feature map on each channel are multiplied point by point to generate a weighted feature map (equivalent to the weighted feature map mentioned above).

[0051] S208 inputs the weighted feature map into the pre-trained third neural network and outputs multiple bounding boxes.

[0052] It should be noted that the third neural network model mentioned above can be a neural network model. The bounding boxes mentioned above can be the identified head region, for example, a rectangle parallel to the axis.

[0053] In one instance, the center point of the detection box can be at the center of the head area.

[0054] S210, Based on the coordinates in the heatmap that have a score greater than a preset threshold, determine the detection box with the highest coordinate score among multiple detection boxes as the detection result.

[0055] It should be noted that the above score can be the score of a point and its four neighborhoods.

[0056] For example, within the generated heatmap, the coordinates of all points with scores greater than the threshold T are taken, and among the many bounding boxes, the bounding box with the highest score at its coordinate location and its four neighboring areas is selected as the final detection result.

[0057] This disclosure, in a head detection scenario, transforms ordinary rectangular annotations to generate a head location heatmap and trains a first neural network. The first neural network predicts the head location and is fused with a feature map from a second neural network to implement an attention mechanism, aiding in the training of the second and third neural networks. The first neural network directly obtains the final bounding boxes from candidate bounding boxes, replacing the non-maximum suppression process and accelerating post-processing. This disclosure, by predicting the head location using a first neural network heatmap and fusing it with a feature map from a second neural network to implement an attention mechanism, can obtain more accurate head bounding boxes, thus contributing to improved accuracy in detecting small-sized heads.

[0058] In one embodiment of this disclosure, such as Figure 3As shown, the target detection method provided in this embodiment can train a first neural network through the following steps to accurately predict the center position of a person's head in an image:

[0059] S302, acquire the target image;

[0060] S304, Mark the human head in the target image with a rectangular box to determine the annotation box, wherein the annotation box includes coordinate information and the center of the annotation box corresponds to the center of the human head region;

[0061] S306, Determine the heat map based on the output of the target image after it is input into the first neural network;

[0062] S308, Determine the true heatmap based on the center point marked in the rectangle;

[0063] S310, train the first neural network based on the heatmap and the true heatmap.

[0064] In a specific example, the data processing for the target image includes acquisition, annotation, and ground truth heatmap generation. The acquisition and annotation process involves creating a dataset using images collected on-site and online (equivalent to the target image mentioned above). All head regions within the dataset are annotated with rectangular boxes, ensuring the center point of each box is centered within the head region. The coordinates of these rectangular boxes are directly used for training the backbone network (equivalent to the second neural network mentioned above) and the detection network (equivalent to the third neural network mentioned above). Furthermore, the ground truth heatmap is generated using these box annotations. First, the original image and its corresponding annotations are scaled, ensuring the scaled size matches the heatmap generated by the auxiliary network (equivalent to the first neural network mentioned above). Then, the center point coordinates of the scaled rectangular boxes are assigned as P1, P2, P3, etc., and the pixel values ​​of each center point and its four neighboring pixels are set to 1, while the pixel values ​​of other locations are set to 0, generating the ground truth heatmap H_GT.

[0065] In one embodiment of this disclosure, such as Figure 4 As shown, the target detection method provided in this embodiment can output a feature map through the following steps, which can accurately determine the feature map size:

[0066] S402, input the target image into the second neural network, and output a feature map with the same size as the heatmap.

[0067] In one embodiment of this disclosure, such as Figure 5 As shown, the target detection method provided in this embodiment can train a third neural network through the following steps to accurately calculate the weighted feature map of the target image:

[0068] S502, the feature map and the heat map are fused by weighting to determine the weighted feature map;

[0069] S504, train the third neural network based on the weighted feature map.

[0070] In a specific example, feature fusion is performed by weighting the feature map and the heatmap to determine the weighted feature map. This includes multiplying the heatmap point by point with the feature map on each channel, weighting the feature map on each channel, and determining the weighted feature map. The heatmap includes one channel, and the feature map includes at least one channel.

[0071] For example, the heatmap H has 1 channel and the feature map has C channels, including three channels: C1, C2, and C3. When performing feature fusion, the heatmap H is weighted and calculated with feature maps C1, C2, and C3 respectively to determine the weighted feature map. The weights of feature maps C1, C2, and C3 can be allocated according to the actual situation.

[0072] In a specific example, the target detection method provided in this disclosure embodiment further includes: updating the parameters of the second neural network and the third neural network according to the backpropagation algorithm.

[0073] In a specific example, the target detection method provided in this disclosure embodiment further includes: a first neural network deployed in a first thread; and a second neural network and a third neural network deployed in a second thread.

[0074] The first and second neural networks of this disclosure are decoupled, allowing for parallel processing during deployment and improving processing efficiency.

[0075] Figure 6 This diagram illustrates the overall training of a small-sized head detection method based on an attention mechanism, as shown in an embodiment of this disclosure.

[0076] like Figure 6 As shown, the model training process includes the training process of the auxiliary network (equivalent to the first neural network mentioned above), the feature fusion process (fusion of heatmap and feature map features), and the simultaneous training process of the model's backbone network (equivalent to the second neural network mentioned above) and the final detection network (equivalent to the third neural network mentioned above).

[0077] The model training process includes the training process of the auxiliary network, the feature fusion process, and the simultaneous training process of the model's backbone network (equivalent to the second neural network mentioned above) and the final detection network (equivalent to the third neural network mentioned above). First, the input image 61 (equivalent to the target image mentioned above) is input into the auxiliary network and the backbone network respectively to obtain the heat map H and the feature map F. Then, the heat map H and the feature map F are fused to generate the weighted feature map F1 (equivalent to the weighted feature map mentioned above), and then sent to the detection network.

[0078] The inference process of the model involves, after the network model is trained, obtaining a large number of candidate boxes from the detection network output. The final detection box is then selected directly from the locations of points in the heatmap that exceed a threshold (e.g., (x0, y0), (x1, y1)) and their surrounding areas. (When selecting points in the heatmap, it's important to note that if a point and all points within its four-neighborhood exceed the threshold, only the largest one is selected). The selection rule is to choose the detection box with the highest score among that point and its four neighbors as the final result. This replaces the non-maximum suppression process, and the threshold used in this process can be adjusted according to the actual situation.

[0079] Figure 7 This diagram illustrates the auxiliary network training for a small-sized head detection method based on an attention mechanism, according to an embodiment of this disclosure.

[0080] like Figure 7 As shown, the auxiliary network (the first neural network) consists entirely of convolutions and max pooling (i.e., Figure 7 In section 71 (including convolution and max pooling), the number of network layers (greater than 5) and the size of the input image can be set according to the actual situation. The auxiliary network finally generates a heatmap H after passing through the fully connected layer 72, and compares it point by point with the true value H_GT of the heatmap, and calculates the cross-entropy loss point by point.

[0081] In a specific example, firstly, collect as many surveillance images as possible of the actual scene to be detected, ensuring at least 3000 images. Next, label the head regions in the images using parallel-axis rectangles. Then, scale the original image and the corresponding labels, ensuring the scaled size matches the heatmap generated by the auxiliary network (e.g., if the auxiliary network generates a heatmap of 80*60 pixels, the scaled image should also be 80*60 pixels). Then, take the center coordinates of the scaled rectangles as P1, P2, P3, etc., and set the pixel values ​​of each center point and its four neighbors to 1, while setting the pixel values ​​of other locations to 0, generating the ground truth heatmap H_GT. The heatmap can be saved as a single-channel binary image or as coordinate points for training the auxiliary network.

[0082] Next is the model training section, where the auxiliary network module and the backbone and detection network modules are trained separately. The auxiliary network consists entirely of convolutional and max-pooling layers. The number of network layers and the size of the input image can be freely determined according to the actual situation. After passing through fully connected layers, the auxiliary network generates a heatmap, which is then compared point-by-point with the ground truth heatmap, and the cross-entropy loss is calculated point-by-point. After the auxiliary network is trained, the backbone and detection networks are trained. The backbone network also consists of multiple convolutional and pooling layers. The size of the output feature map of the backbone network should be consistent with that of the auxiliary network. It is responsible for extracting image features. Finally, the feature map calculated by the backbone network is multiplied point-by-point with each channel of the heatmap to weight the feature map of each channel, enhancing the response of the head region and suppressing the response of the background. Finally, the fused feature map is input into the final detection network, and the detection network is trained. During backpropagation, only the parameters of the detection network and the backbone network are updated.

[0083] During model inference, the input image is first fed into the auxiliary network and the backbone network respectively to obtain heatmaps and feature maps. Then, the heatmap and feature map are fused and fed into the detection network. After the detection network outputs a large number of candidate boxes, the final detection box is directly selected from the locations of points in the heatmap that exceed a threshold, and their surrounding areas. (When selecting points in the heatmap, it is important to note that if a point and all points in its four-neighborhood exceed the threshold, only the largest one is selected). The selection rule is to choose the detection box with the highest score among the point and its four neighbors as the final result. This replaces the non-maximum suppression process, and the threshold used in this process can be adjusted according to the actual situation.

[0084] Finally, during deployment, the weight files of the auxiliary network, backbone network, and detection network are first converted to the format required by the corresponding chip manufacturer. Each chip and device has its own conversion method, and the parameter precision can be appropriately reduced to improve inference speed. The algorithm proposed in this method can decouple the auxiliary network and backbone network, so they can be computed in parallel during inference. The algorithm uses two threads during deployment: the auxiliary network is deployed on thread 1 (equivalent to the first thread mentioned above), and the backbone network and detection network are deployed on thread 2 (equivalent to the second thread mentioned above). After thread 1 finishes running, the auxiliary network shares the heatmap and the coordinates of the points where the response is greater than the threshold with thread 2. After the backbone network finishes its calculation, it will use the information shared by thread 2 for subsequent calculations.

[0085] Based on the same inventive concept, this disclosure also provides a small-sized head detection device based on an attention mechanism, as described in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0086] Figure 8 This illustration shows a schematic diagram of a small-sized head detection device based on an attention mechanism according to an embodiment of the present disclosure, such as... Figure 8 As shown, the device includes: a first neural network module 81, a second neural network module 82, a feature fusion module 83, a third neural network module 84, a detection result determination module 85, a backpropagation module 86, and a network deployment module 87.

[0087] The system includes a first neural network module 81, which inputs the target image into a pre-trained first neural network and outputs a heatmap; a second neural network module 82, which inputs the target image into a second neural network and outputs a feature map; a feature fusion module 83, which fuses the features of the heatmap and the feature map to determine a weighted feature map; a third neural network module 84, which inputs the weighted feature map into a pre-trained third neural network and outputs multiple detection boxes; and a detection result determination module 85, which determines the detection box with the highest coordinate score among the multiple detection boxes as the detection result based on the coordinates in the heatmap that have a score greater than a preset threshold.

[0088] In one embodiment of this disclosure, the first neural network module 81 is further configured to: acquire a target image; mark the human head in the target image with a rectangular box to determine the marking box, wherein the marking box includes coordinate information and the center of the marking box corresponds to the center of the human head region; determine a heat map based on the output of the target image after inputting it into the first neural network; determine a ground truth heat map based on the center point of the rectangular box marking; and train the first neural network based on the heat map and the ground truth heat map.

[0089] In one embodiment of this disclosure, the second neural network module 82 is further configured to: input the target image into the second neural network and output a feature map with the same size as the heatmap.

[0090] In one embodiment of this disclosure, the third neural network module 84 is further configured to: perform feature fusion by weighting the feature map and the heatmap to determine a weighted feature map; and train the third neural network based on the weighted feature map.

[0091] In one embodiment of this disclosure, the feature fusion module 83 is further configured to: multiply the heatmap by the feature map on each channel point by point, weight the feature map on each channel, and determine the weighted feature map, wherein the heatmap includes one channel and the feature map includes at least one channel.

[0092] In one embodiment of this disclosure, the small-sized head detection device based on the attention mechanism further includes a backpropagation module 86 for updating the parameters of the second neural network and the third neural network according to the backpropagation algorithm.

[0093] In one embodiment of this disclosure, the aforementioned small-sized head detection device based on the attention mechanism further includes a network deployment module 87 for: deploying a first neural network in a first thread; and deploying a second neural network and a third neural network in a second thread.

[0094] It should be noted that the first neural network module, the second neural network module, the feature fusion module, the third neural network module, and the detection result determination module mentioned above correspond to S202 to S210 in the method embodiment. The examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system such as a set of computer-executable instructions.

[0095] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0096] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0097] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0098] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0099] For example, the processing unit 910 can perform the following steps in the above method embodiment: inputting the target image into a pre-trained first neural network and outputting a heatmap, wherein the heatmap reflects the location of the human head; inputting the target image into a second neural network and outputting a feature map; fusing the features of the heatmap and the feature map to determine a weighted feature map; inputting the weighted feature map into a pre-trained third neural network and outputting multiple detection boxes; and determining the detection box with the largest coordinate score among the multiple detection boxes as the detection result based on the coordinates in the heatmap that have a score greater than a preset threshold.

[0100] For example, the processing unit 910 can perform the following steps in the above method embodiment: acquiring a target image; marking the human head in the target image with a rectangular box to determine the marking box, wherein the marking box includes coordinate information and the center of the marking box corresponds to the center of the human head region; determining a heat map based on the output of the target image after inputting it into the first neural network; determining a ground truth heat map based on the center point of the rectangular box marking; and training the first neural network based on the heat map and the ground truth heat map.

[0101] For example, the processing unit 910 can perform the following steps in the above method embodiment: inputting the target image into the second neural network and outputting a feature map with the same size as the heatmap.

[0102] For example, the processing unit 910 may perform the following steps in the above method embodiment: perform feature fusion by weighting the feature map and the heat map to determine a weighted feature map; and train a third neural network based on the weighted feature map.

[0103] For example, the processing unit 910 may perform the following steps in the above method embodiment: multiply the heat map point by point with the feature map on each channel, weight the feature map on each channel, and determine the weighted feature map, wherein the heat map includes one channel and the feature map includes at least one channel.

[0104] For example, the processing unit 910 can perform the following steps in the above method embodiment: update the parameters of the second neural network and the third neural network according to the backpropagation algorithm.

[0105] For example, the processing unit 910 can execute the following steps of the above method embodiment: the first neural network is deployed in the first thread; the second neural network and the third neural network are deployed in the second thread.

[0106] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0107] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0108] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0109] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0110] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0111] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described attention-based small head detection method.

[0112] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. Figure 10 This diagram illustrates a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 10 As shown, the computer-readable storage medium 1000 stores a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0113] For example, when the program product in this embodiment is executed by a processor, it implements the following steps: inputting a target image into a pre-trained first neural network and outputting a heatmap, wherein the heatmap reflects the location of a person's head; inputting the target image into a second neural network and outputting a feature map; fusing the heatmap and the feature map features to determine a weighted feature map; inputting the weighted feature map into a pre-trained third neural network and outputting multiple detection boxes; and determining the detection box with the largest coordinate score among the multiple detection boxes as the detection result based on the coordinates in the heatmap that have a score greater than a preset threshold.

[0114] For example, when the program product in this embodiment is executed by the processor, it implements the following steps: acquiring a target image; marking the human head in the target image with a rectangular box to determine the marking box, wherein the marking box includes coordinate information and the center of the marking box corresponds to the center of the human head region; determining a heat map based on the output of the target image after inputting it into the first neural network; determining a ground truth heat map based on the center point of the rectangular box marking; and training the first neural network based on the heat map and the ground truth heat map.

[0115] For example, when the program product in this embodiment is executed by a processor, it implements the following steps: inputting a target image into a second neural network and outputting a feature map of the same size as the heatmap.

[0116] For example, when the program product in this embodiment is executed by the processor, it implements the following steps: performing feature fusion of feature map and heatmap through weighting to determine weighted feature map; and training a third neural network based on the weighted feature map.

[0117] For example, when the program product in this embodiment is executed by the processor, it implements the following steps: multiplying the heatmap point by point with the feature map on each channel, weighting the feature map on each channel, and determining the weighted feature map, wherein the heatmap includes one channel and the feature map includes at least one channel.

[0118] For example, when the program product in this embodiment is executed by a processor, it implements the following steps: updating the parameters of the second neural network and the third neural network according to the backpropagation algorithm.

[0119] For example, when the program product in this embodiment is executed by a processor, it implements the following steps: a first neural network is deployed in a first thread; a second neural network and a third neural network are deployed in a second thread.

[0120] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0122] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0123] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0124] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0125] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0126] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0127] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for detecting small-sized heads based on an attention mechanism, characterized in that, include: The target image is input into a pre-trained first neural network, which outputs a heat map, wherein the heat map reflects the location of the human head, and the target image is an image taken in the context of a construction site safety helmet scene; The target image is input into the second neural network, which outputs a feature map. The heatmap and the feature map features are fused to determine a weighted feature map; The weighted feature map is input into a pre-trained third neural network, which outputs multiple detection boxes. Based on the coordinates in the heat map that have a score greater than a preset threshold, the detection box with the highest coordinate score among multiple detection boxes is determined as the detection result. The step of determining the detection box with the largest coordinate score among multiple detection boxes based on the coordinates of points with scores greater than a preset threshold in the heat map includes: taking the coordinates of all points with scores greater than the preset threshold in the heat map, and selecting the detection box with the largest score at the coordinate position and its four neighboring areas as the final detection result.

2. The small-sized head detection method based on attention mechanism according to claim 1, characterized in that, The pre-trained first neural network includes: Acquire the target image; The head in the target image is marked with a rectangular box to determine the marking box. The marking box includes coordinate information and the center of the marking box corresponds to the center of the head area. A heatmap is determined based on the output of the first neural network after the target image is input; Determine the true heatmap based on the center point marked by the rectangle; The first neural network is trained based on the heatmap and the ground truth heatmap.

3. The small-sized head detection method based on attention mechanism according to claim 1, characterized in that, The step of inputting the target image into the second neural network and outputting a feature map includes: The target image is input into the second neural network, which outputs a feature map of the same size as the heatmap.

4. The small-sized head detection method based on attention mechanism according to claim 1, characterized in that, The pre-trained third neural network includes: The feature map and the heat map are fused by weighting to determine the weighted feature map; The third neural network is trained based on the weighted feature map.

5. The small-sized head detection method based on an attention mechanism according to claim 4, characterized in that, The feature map and the heatmap are fused using a weighted method to determine the weighted feature map, which includes: The heatmap is multiplied point by point with the feature map on each channel, and the feature map on each channel is weighted to determine the weighted feature map. The heatmap includes one channel, and the feature map includes at least one channel.

6. The small-sized head detection method based on attention mechanism according to claim 1, characterized in that, The method further includes: The parameters of the second and third neural networks are updated according to the backpropagation algorithm.

7. The small-sized head detection method based on attention mechanism according to claim 1, characterized in that, The method further includes: The first neural network is deployed in the first thread; The second and third neural networks are deployed in the second thread.

8. A small-sized head detection device based on an attention mechanism, characterized in that, include: The first neural network module is used to input the target image into the pre-trained first neural network and output a heat map, wherein the heat map reflects the location of the human head, and the target image is an image taken in the context of a construction site safety helmet scene; The second neural network module is used to input the target image into the second neural network and output a feature map. The feature fusion module is used to fuse the features of the heatmap and the feature map to determine a weighted feature map; The third neural network module is used to input the weighted feature map into a pre-trained third neural network and output multiple detection boxes. The detection result determination module is used to determine the detection box with the largest coordinate score among multiple detection boxes as the detection result based on the coordinates in the heat map that have a score greater than a preset threshold. The detection result determination module is further configured to take the coordinates of all points with scores greater than a preset threshold within the heatmap, and select the detection box with the highest score at the coordinate position and its four neighboring areas as the final detection result.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the attention-based small head detection method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the small-sized head detection method based on the attention mechanism as described in any one of claims 1 to 7.

Images