A method and system for human target recognition

Abstract

This invention provides a method and system for human target recognition, employing an improved lightweight Yolov5s network target detection model. The method includes the following steps: real-time acquisition of human target motion video; inputting each frame of the original human target motion image from the human target motion video into the model; preprocessing; performing multi-scale fusion; and constructing the Ciou loss function. Loss The aspect ratio γ function is used to evaluate the human target recognition and localization effect of the predicted bounding box image and the ground truth bounding box image after multi-scale fusion. If Ciou Loss If γ < 0.2 and γ < 0.03, then human target recognition is complete, and the recognized human target image is output; otherwise, the above steps are repeated. This invention can effectively reduce the computational load of human target recognition and detection while maintaining high precision and recall. The improved Yolov5s model of this invention shows improvements in precision, recall, detection speed, and GPU performance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of digital image processing and pattern recognition, and particularly relates to a human target recognition method and system. BACKGROUND

[0002] In recent years, the field of video images has accelerated development, and it is easy to frame pedestrians on monitoring equipment, but it is more difficult to obtain more information about the characters to judge the behavior of the characters, because there are many factors affecting the monitoring, such as environmental interference, hardware device computing power, non-uniform standards for measuring and obtaining information, etc. In addition, human posture estimation and behavior judgment are used in various aspects of life places, and the demand for monitoring market is increasing.

[0003] At present, the proposal of smart city and digital economy in China will drive the great development of China's economy in the future deployment and planning of smart city. In the process of promoting urban intelligence, the intelligent hardware industry in various fields has developed rapidly, and a large number of intelligent software or hardware have flowed into urban families, such as smart home, Internet camera, interactive intelligent robot, etc. At the same time, the most popular among many intelligent devices is the home networking camera, because the resolution of the camera in the home environment does not need to be too high, the price is low, and the operation steps of the home monitoring function are simple. A simple indoor monitoring system only needs a camera, a storage hard disk and a network connection. However, the most core technology is machine vision, and the main body of the home monitoring device is often a person. After detecting the state of the person, the person can interact with the machine. How to judge the state of the person is the first problem to be solved.

[0004] The human target recognition detection algorithm in the prior art occupies a large amount of computing performance of the intelligent chip, because the human target recognition detection is the first part of the whole monitoring system. It first inputs the whole picture shot by the camera into the detection algorithm. It is difficult for a small intelligent chip with small computing performance to monitor the person in real time in video shooting. With the popularity of deep learning, the algorithm model in the target detection field is introduced into the video monitoring field, but the complex model cannot be all suitable for the intelligent chip in the market. How to select the appropriate algorithm to achieve real-time effect under the condition of limited cost is the focus of competition among various manufacturers.

[0005] Therefore, there is an urgent need for a human target recognition method and system which can effectively reduce the computing amount of human target recognition detection, and has high precision rate and recall rate. SUMMARY

[0006] The present application aims at the above-mentioned defects, and provides a target recognition method and system. The present application improves the backbone network layer into a gconv layer and a G_CSP improved residual layer, and uses a depth separable convolution and a group convolution with g groups in the preprocessing of the picture obtained after the group convolution of the backbone network layer and the Neck feature processing layer and the multi-scale fusion process, reduces the parameter quantity of the overall network through the decomposition step of multiplication, reduces the calculation amount, retains the SPP spatial pyramid layer in the Neck feature processing layer, and increases the CBAM convolution attention layer using the mean pooling and the maximum pooling between the SPP spatial pyramid layer and the second CBL layer of the Neck feature processing layer, which considers the global field of view and highlights the feature details of the obvious target, and constructs a Ciou Loss loss function Ciou Loss and a width-height ratio gamma function, optimizes the output structure evaluation index of the prediction box. Through experimental comparison, the improved Yolov5s model has improved in precision, recall rate, detection speed and GPU performance.

[0007] The calculation amount of human target recognition detection can be effectively reduced, and the precision and recall rate are high

[0008] The present application provides the following technical scheme: a human target recognition method using an improved lightweight Yolov5s network target detection model, which comprises the following steps:

[0009] 1) real-time acquisition of a human target motion video, wherein each frame of original human target motion picture in the human target motion video is input into the model;

[0010] 2) preprocessing of the original human target motion picture;

[0011] 3) multi-scale fusion of the preprocessed human target motion picture obtained in the step 2);

[0012] 4) construction of a Ciou loss function Ciou Loss and a width-height ratio gamma function;

[0013] 5) evaluation of the human target recognition positioning of the image in the prediction box and the image in the real box after the multi-scale fusion in the step 3) using the Ciou loss function Ciou Loss value and the width-height ratio gamma function value, if Ciou Loss <0.2 and gamma <0.03, the human target recognition is completed, and the recognized human target image is output, otherwise, the steps 1)-3) are repeated.

[0014] Further, the preprocessing of the original human target motion picture in the step 2) comprises:

[0015] 2.1) Dimensionality reduction is performed on the original human target motion picture of size X X X 3 collected in step 1) to obtain a feature map of size The information on the original picture is saved in the number of channels;

[0016] 2.2) The picture obtained in step 2.1) is subjected to two-group convolution, and the final feature map of the backbone is obtained by splicing;

[0017] 2.3) The final feature map obtained by splicing in step 2.2) is subjected to depth separable convolution and group convolution with g groups, and the number of parameters of the overall network is reduced through the decomposition step of multiplication;

[0018] 2.4) The steps 2.2)-2.3) are repeated twice to complete the preprocessing of the original human target motion picture.

[0019] Further, the size of X in step 2.1) is 80-640.

[0020] Further, the multi-scale fusion of the human target motion picture obtained in step 2) in step 3) includes the following steps:

[0021] 3.1) Two-group convolution is performed on the preprocessed human target motion picture, and the final feature map of the Neck feature processing is obtained by splicing;

[0022] 3.2) Different sizes of pooling windows are used to pool the data respectively, and then the pooling results of different pooling windows are spliced for multi-scale fusion;

[0023] 3.3) Depth separable convolution and group convolution with g groups are performed on the image after multi-scale fusion in step 3.2), and the number of parameters of the overall network is reduced through the decomposition step of multiplication;

[0024] 3.4) The picture obtained after processing in step 3.3) is simultaneously subjected to mean pooling and maximum pooling to construct a CBAM convolution attention pooling model, and the input feature map of size H X W X C after processing in step 3.3) is pooled into an output feature map of size H X W X 1;

[0025] 3.5) The output is obtained after ordinary convolution is performed on the output feature map of size H X W X 1 obtained in step 3.4).

[0026] Further, the CBAM convolution attention pooling model constructed in step 3.4) has the following formula:

[0027] M s (F)=σ(f 7×7([AvgPool(F) ; MaxPool(F)])) ;

[0028] wherein, σ represents sigmoid activation function, f 7×7 represents 7x7 convolution kernel convolution operation, AvgPool(F) represents average pooling, MaxPool(F) represents maximum pooling, F is the input feature map of step 3.3) processing size HxWxC.

[0029] Further, the size of the pooling window in step 3.2) is 5x5, 9x9 and 13x13.

[0030] Further, the depth separable convolution includes: converting the input feature map with size D F x D F x N into N standard convolution kernels D K x D K x M, and then each of the standard convolution kernels is split into M depth convolution with size D K x D K x 1 and 1x1xM point convolution, and the convolution calculation formula is as follows:

[0031] (D K · D K · M· D F · D F + M· D F · D F )· N;

[0032] Wherein, D K · D K · M· D F · D F represents the depth convolution calculation, M· N· D F · D F represents the point convolution calculation, D K represents the convolution kernel size, M is the number of input feature map channels, and N is the number of output feature layer channels.

[0033] Further, the g-group group convolution is to divide the input feature map with size HxWxC1 into g groups according to the number of channels, and then obtain the group input feature map with size ;

[0034] The convolution kernel size of the input feature map with size HxWxC1 is The size of the group input feature map is The dimension of the final feature map spliced is HxWxC2, that is, the dimension of the backbone final feature map spliced by the step 2.2) and the step 3.1) and the dimension of the Neck feature processing final feature map are both HxWxC2; wherein H is the height of the input feature map, W is the width of the input feature map, C1 is the channel number of the input feature map, C2 is the channel number of the final feature map, K is the length value and width value of the convolution kernel of the grouped convolution;

[0035] The calculation formula of the grouped convolution is

[0036] Further, the Ciou loss function Ciou in the step 4) is Loss As follows:

[0037]

[0038] Wherein, Distance_O 2 represents the Euclidean distance of two center points of the prediction box and the real box, Distance_C 2 represents the diagonal distance of the minimum circumscribed rectangle of the prediction box and the real box; gamma is the width-height ratio, Iou is the intersection ratio of the prediction box and the real box,

[0039] The width-height ratio gamma function is as follows:

[0040]

[0041] Wherein, w gt represents the width of the real box, h gt represents the height of the real box, w p represents the width of the prediction box, h p represents the height of the prediction box.

[0042] The application also provides a human target recognition system, comprising a human target motion picture acquisition module, a human target picture preprocessing module, a human target picture multi-angle fusion module and a human target recognition control module.

[0043] The human target motion picture acquisition module is used for acquiring human target motion video in real time, and inputting each frame of original human target motion picture in the acquired human target motion video into a model;

[0044] The human target motion picture preprocessing module is used for preprocessing the original human target motion picture;

[0045] The human target picture multi-angle fusion module is used for multi-scale fusion of the human target motion picture after preprocessing;

[0046] The human target recognition control module is used to construct the Ciou loss function. Loss The aspect ratio γ function is used to evaluate whether the prediction results of each point in the predicted bounding box after multi-scale fusion meet the standard, and then control whether to output the recognized human target image.

[0047] The beneficial effects of this invention are as follows:

[0048] 1. Since this paper improves the ordinary convolution operation to depthwise separable convolution and grouped convolution, the overall number of network layers is greatly increased. In order to solve the gradient vanishing problem caused by the increase in the number of network layers, a residual network structure is used to alleviate the problem. The first part uses a bottleneck structure, which first increases the dimension and then decreases the dimension. The second part uses depthwise separable convolution and grouped convolution with a group number of g groups. This not only takes into account global information, but also adjusts the number of feature channels to facilitate the generation of the final integer layer.

[0049] 2. The human target recognition method provided by this invention, in the preprocessing and multi-scale fusion of the original human target motion image, utilizes depthwise separable convolution and grouped convolution methods in steps 2.2) and 2.3) to reduce the network parameters in the Yolov5 algorithm. It replaces the CSPNet in the backbone feature network of the existing technology with the G_CSP network of the improved residual layer, thereby improving the ordinary convolution used by CSPNet into a combination of depthwise separable convolution and grouped convolution with g groups to process image data. Simultaneously, it retains the SPP spatial pooling pyramid layer in the Neck feature processing layer, and adds a CBAM convolutional attention layer that simultaneously uses mean pooling and max pooling between the SPP spatial pooling pyramid layer and the second CBL layer of the Neck feature processing layer. This considers both the global field of view and highlights the obvious feature details of the target. The method also constructs a Ciou loss function. Loss A width-to-height ratio γ function was constructed to optimize the output structure evaluation metric of the predicted bounding boxes. Experimental comparisons showed that the improved Yolov5s model achieved improvements in accuracy, recall, detection speed, and GPU performance.

[0050] Experiments have shown that, compared to existing YOLOv5s models, the improved YOLOv5s model based on this invention demonstrates improvements in accuracy, recall, detection speed, and GPU performance.

[0051] 3. This invention uses a constructed Ciou loss function to determine whether the human target recognition result meets the standard. Loss And the Ciou loss function, constructed from the aspect ratio γ function related to the width and height of the anchor frame. LossThe ratio of the Euclidean distance of the two frame (real frame and predicted frame) centers to the diagonal distance of the minimum circumscribed rectangle is adopted, and the smaller the value is, the closer the predicted frame center is to the real frame center, and the better the regression effect is judged, and the denominator is the diagonal distance of the minimum circumscribed rectangle, and the ratio is normalized to facilitate regression calculation; the width-height ratio gamma constructed by the application to judge whether the human target recognition result meets the standard is related to the width and height of the anchor frame, if the width and height of the predicted frame are similar to the width and height of the real frame, the smaller the gamma value is, the better the prediction effect is, and the arctan function is used to limit the value between 0 and 1, which is convenient for regression calculation.

[0052] The human target recognition effect is discriminated whether it meets the output standard through the Ciou loss function Ciou Loss and the width-height ratio gamma function, which improves the accuracy of human target recognition and avoids the misjudgment of the predicted frame caused by using the unoptimized Giou loss function, thereby avoiding the occurrence of the increase of calculation noise. BRIEF DESCRIPTION OF DRAWINGS

[0053] In the following, the application will be described in more detail based on embodiments and with reference to the accompanying drawings. In which:

[0054] Figure 1 The flowchart of the human target recognition method provided by the application;

[0055] Figure 2 The structure diagram of the backbone neural network including the backbone network layer and the Neck feature processing layer in the human target recognition method provided by the application;

[0056] Figure 3 The structure diagram of the gconv convolution layer in the human target recognition method provided by the application;

[0057] Figure 4 The attention mechanism structure diagram of the CBAM convolution attention pooling model;

[0058] Figure 5 The specific structure diagram of the spatial attention mechanism of the CBAM convolution attention pooling model;

[0059] Figure 6 The method step decomposition diagram of the depth separable convolution;

[0060] Figure 7 The structure diagram of the human target recognition system provided by the application;

[0061] Figure 8 The result diagram of the output predicted frame of the human target recognition in the prior art using the Giou loss function as the judgment;

[0062] Figure 9The Loss variation curve of the improved Yolov5s model after 100 training iterations in Test Example 1 using the method of the present invention;

[0063] Figure 10 The image shows the iterative graphs of the loss functions for the three models, YOLOv5s, YOLOv5m, and YOLOv5s, in Test Example 2 of this invention.

[0064] Figure 11 This is a graph showing the performance index (PR) curve of the improved Yolov5s model in Test Example 2 of this invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] like Figure 1 The diagram shown is a flowchart of a human target recognition method provided by the present invention. The method provided by the present invention uses an improved lightweight Yolov5s network target detection model, and the method includes the following steps:

[0067] 1) Real-time acquisition of human target motion video, with each frame of the original human target motion image in the human target motion video input into the model; the original human target motion images come from each frame of the human target motion video captured by the camera.

[0068] 2) Preprocess the original human target motion images;

[0069] 3) Perform multi-scale fusion on the preprocessed human target motion images obtained in step 2) to further improve the diversity and robustness of features;

[0070] 4) Construct the Ciou loss function. Loss and the aspect ratio γ function;

[0071] 5) Evaluate the Ciou loss function (Ciou) for human target recognition and localization using the predicted bounding box image and the ground truth bounding box image after multi-scale fusion in step 3). Loss Value and aspect ratio γ function value, if Ciou Loss If γ < 0.2 and γ < 0.03, then human target recognition is completed and the recognized human target image is output; otherwise, repeat steps 1)-3).

[0072] As a preferred embodiment of the present application, step 2) pre-processes the original human target motion picture, which includes:

[0073] 2.1) Dimension reduction is performed on the original human target motion picture collected in step 1) with a size of X X X 3 to obtain a feature map with a size of The information on the original picture is saved in the number of channels to enhance the feature information in the down-sampling;

[0074] 2.2) Group convolution is performed on the picture obtained in step 2.1), and the final feature map of the backbone is obtained by splicing;

[0075] 2.3) Depth separable convolution and group convolution with g groups are performed on the final feature map obtained by splicing in step 2.2), and the number of parameters of the overall network is reduced through the decomposition step of multiplication;

[0076] 2.4) After repeating steps 2.2)-2.3) twice, the pre-processing of the original human target motion picture is completed.

[0077] As a preferred embodiment of the present application, step 2) pre-processes the original human target motion picture using a backbone network layer. Preferably, as shown in Figure 2 , the backbone network layer sequentially includes a first CBL layer of the backbone, a first gconv convolutional layer of the backbone, a first G_CSP improved residual layer of the backbone, a second gconv convolutional layer of the backbone, a second G_CSP improved residual layer of the backbone, a third gconv convolutional layer of the backbone, and a third G_CSP improved residual layer of the backbone.

[0078] Further, further preferably, step 2.1) is completed by the first CBL layer of the backbone, step 2.2) is completed by the first gconv convolutional layer of the backbone, step 2.3) is completed by the first G_CSP improved residual layer of the backbone, the first repetition of steps 2.3)-2.3) in step 2.4) is sequentially completed by the second gconv convolutional layer of the backbone and the second G_CSP improved residual layer of the backbone, and the second repetition of steps 2.3)-2.3) in step 2.4) is completed by the third gconv convolutional layer of the backbone and the third G_CSP improved residual layer of the backbone.

[0079] Preferably, in step 2.2), group convolution is performed on the picture obtained in step 2.1), as shown in Figure 3 , the group convolution uses two CBL layers to reduce the parameter amount of ordinary convolution which only needs one CBL layer, so that the parameter amount of each CBL layer for convolution is generally that of ordinary convolution, and then a second convolution similar to the current operation is performed to generate an imaginary feature map, and then the final feature map of the backbone is obtained by splicing; all gconv convolutional layers in the present application adopt the structure as shown in Figure 3 .

[0080] Preferably, the size of X in step 2.1) is 80-640, that is, the original human target motion picture collected with a size of 608x608x3 can be reduced to 304x304x32, or the original human target motion picture collected with a size of 80x80x3 can be reduced to 40x40x32, or the original human target motion picture collected with a size of 640x640x3 can be reduced to 320x320x32, and the dimension reduction can be performed according to the specific original human target motion picture.

[0081] As another preferred embodiment of the present application, the multi-scale fusion of the pre-processed human target motion picture obtained in step 2) in step 3) comprises the following steps:

[0082] 3.1) performing two-group convolution on the pre-processed human target motion picture to obtain a Neck feature processing final feature map by splicing;

[0083] 3.2) using different sizes of pooling windows to pool the data respectively, and then splicing the pooling results of different pooling windows for multi-scale fusion, which can aggregate deep information; using different sizes of pooling windows, the target deformation has stronger robustness;

[0084] 3.3) performing depth separable convolution and group convolution with g groups on the image after multi-scale fusion in step 3.2), to reduce the number of parameters of the overall network through the decomposition step of multiplication;

[0085] 3.4) using mean pooling and maximum pooling on the picture obtained after step 3.3) to construct a CBAM convolution attention pooling model, pooling the input feature map with a size of HxWxC after step 3.3) into an output feature map with a size of HxWx1, which considers the global view and highlights the obvious feature details of the target, and the double pooling effect is better than the single pooling effect, and the convolution network is used to learn the parameter optimization, and finally weighted to the original feature map;

[0086] 3.5) performing ordinary convolution on the output feature map with a size of HxWx1 obtained in step 3.4) and outputting.

[0087] Preferably, step 3) uses a Neck feature processing layer (Neck network) to perform multi-scale fusion on the pre-processed human target motion picture obtained in step 2). Preferably, as Figure 2As shown, the Neck feature processing layer sequentially includes a fourth gconv convolutional layer, an SPP spatial reddening pyramid layer, a fourth G_CSP improved residual layer, a CBAM convolutional attention layer, and a second CBL layer; furthermore, preferably, the fourth gconv convolutional layer completes step 3.1), the SPP spatial reddening pyramid layer completes step 3.2), the fourth G_CSP improved residual layer completes step 3.3), the CBAM convolutional attention layer completes step 3.4), and the second CBL layer completes step 3.5).

[0088] As another preferred embodiment of the present invention, the formula for the CBAM convolutional attention pooling model constructed in step 3.4) is as follows:

[0089] M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)]));

[0090] Where σ represents the sigmoid activation function, f 7×7 The convolution operation is represented by a 7×7 kernel. AvgPool(F) represents mean pooling, and MaxPool(F) represents max pooling. F is the input feature map of size H×W×C after processing in step 3.3).

[0091] The CBAM convolutional attention pooling model is an evolution of the Squeeze-and-Excitation Network (SElayer), such as... Figure 4 As shown, an additional spatial attention mechanism is introduced, consisting of two steps: channel attention and spatial attention. The channel attention mechanism aims to weight the pixel values ​​on each feature map, so each feature map is pooled, focusing on the weighting of one-dimensional data. The spatial attention mechanism, on the other hand, shifts to learning and optimization at two-dimensional positions, so the entire channel is pooled, and the input feature map of H×W×C is pooled into output features of H×W×1. Both mean pooling and max pooling are used simultaneously, taking into account both the global perspective and highlighting the obvious feature details. The double pooling effect is better than single pooling. After learning and optimizing the parameters through a convolutional network, the final weighted features are added to the original feature map.

[0092] The specific structural diagram of the spatial attention mechanism is as follows: Figure 5 As shown, the spatial attention structure is still similar to the residual network structure. The original feature map is pooled by the maximum value of the channel dimension and average pooled by the average value. Then, the two 1-dimensional feature maps are fused to generate a 2-dimensional feature map. The 2-dimensional feature map is then convolved to generate a 1-dimensional feature map with one channel. Finally, the 2-dimensional feature map is weighted and added to the original feature map to achieve parameter optimization at the 2-dimensional position.

[0093] This invention adds a CBAM convolutional attention layer between the fourth G_CSP improved residual layer and the second CBL layer in the Neck feature processing layer after the SPP spatial reddening pyramid layer, which performs multi-scale fusion. Using the CBAM attention mechanism structure, compared to the attention mechanism of the compressed and activated SElayer network, the CBAM module has only 100 more parameters than the SE module, and compared to the original YOLOv5m network, it only increases the number of parameters by 73,826, a mere 0.5% increase, yet significantly improves the accuracy of target detection. Specific improvements are as follows... Figure 2 As shown, the CBAM structure is embedded in the Neck feature processing layer. Since this feature layer has the lowest resolution, the computational cost will be further reduced compared to other layers. At the same time, due to the network structure of FPN and PAN in YOLOv5 mentioned above, this processed feature layer will be fused with other feature layers multiple times, making fuller use of the weighted feature layer information.

[0094] In the human target recognition method provided by this invention, the SPP spatial pooling pyramid module is retained in the Neck feature processing layer of the backbone neural network. Since SPP obtains local features at different scales by using pooling at different scales, it can effectively increase the acceptance scale of the backbone feature extraction network. By fusing global features with local features at different scales, the detection accuracy is increased with less computational cost.

[0095] Preferably, in the method provided by the present invention, in step 3) of multi-scale fusion of the preprocessed human target motion image obtained in step 2), the pooling window size in step 3.2) is 5×5, 9×9, and 13×13. This paper sets the maximum pooling kernel of the SPP module to 5, 9, and 13 respectively, and fuses the feature maps under these three different pooling conditions with the original feature map. Using pooling operations to scale the candidate boxes can preserve the main feature information and greatly reduce information loss.

[0096] As another preferred embodiment of the present invention, the depthwise separable convolutions in steps 2.3) and 3.3) are as follows: Figure 6 As shown, this includes: inputting a value of D. F ×D F The feature map of ×N is transformed into, for example, Figure 6 (a) shows the N standard convolutional kernels D K ×D K ×M, and then each standard convolutional kernel is split into, as follows Figure 6 (b) shows the M D K ×D K ×1 depthwise convolution and such Figure 6 The formula for the computational complexity of the 1×1×M pointwise convolution shown in (c) is as follows:

[0097] (D K ·D K ·M·D F ·D F +M·D F ·D F )·N;

[0098] Among them, D K ·D K ·M·D F ·D F M·N·D represents the computational cost of depthwise convolution. F ·D F D represents the computational cost of pointwise convolution. K This indicates the kernel size, M is the number of channels in the input feature map, and N is the number of channels in the output feature layer;

[0099] The computational complexity of standard convolution without depthwise separable convolution in existing technologies is as follows:

[0100] D K ·D K ·M·N·D F ·D F ;

[0101] The computational cost of depthwise separable convolution is compared to that of standard convolution, using the following formula:

[0102]

[0103] The comparison calculation formula will include D. K The kernel size is assumed to be 3, and N is the number of channels in the output convolutional layer, which is often a large number. Finally, it is found that the computational cost of standard convolution is N times that of depthwise separable convolution. It can be seen that in the human target method provided in this application, steps 2.3) and 3.3) in the two preferred embodiments respectively adopt depthwise separable convolution, which can reduce the computational cost of CPU for human target recognition.

[0104] In another preferred embodiment of the present invention, the grouped convolution with g groups in steps 2.3) and 3.3) involves dividing the input feature map of dimension H×W×C1 into g groups according to the number of channels, thereby obtaining a convolution with dimension H×W×C1. The input feature map is grouped;

[0105] The kernel size of the input feature map with dimensions H×W×C1 is The dimension of the input feature map after grouping is The final feature map obtained by splicing has a dimension of H×W×C2, which is the dimension of the final feature map of the backbone obtained by splicing in steps 2.2) and 3.1) and the dimension of the final feature map of the Neck feature processing. Wherein, H is the height of the input feature map, W is the width of the input feature map, C1 is the number of channels of the input feature map, C2 is the number of channels of the final feature map, and K is the length and width of the convolution kernel of the grouped convolution.

[0106] The formula for calculating grouped convolution is: It can be seen that the number of parameters in grouped convolution is the same as that in standard convolution.

[0107] If the number of groups g equals 1, it evolves into a normal convolution operation. The binary convolution performed in steps 2.2) and 3.1) of the method provided in this invention is the group convolution performed when the number of groups g equals 2. The binary convolution performed in steps 2.2) and 3.1), and the group convolution with the number of groups g performed in steps 2.3) and 3.3) are structurally simple to calculate, meet the requirements of parallel computing on hardware GPUs, and greatly reduce the computation time.

[0108] As another preferred embodiment of the present invention, the Ciou loss function in step 4) Loss as follows:

[0109]

[0110] Among them, Distance_O 2 Distance_C represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. 2 This represents the diagonal distance between the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box, γ is the aspect ratio, and Iou is the cross-linking ratio between the predicted bounding box and the ground truth bounding box.

[0111] The aspect ratio γ function is as follows:

[0112]

[0113] Among them, w gt h represents the width of the actual bounding box. gt w represents the height of the actual bounding box p h represents the width of the prediction box. p This indicates the height of the prediction box.

[0114] The present invention uses the Ciou loss function (Ciou) to determine whether the human target recognition result conforms to the standard. LossThe method uses the ratio of the Euclidean distance between the centers of the two bounding boxes (the ground truth bounding box and the predicted bounding box) to the diagonal distance of the minimum bounding rectangle. A smaller value indicates that the center of the predicted bounding box is closer to the center of the ground truth bounding box, indicating a better regression effect. The denominator is the diagonal distance of the minimum bounding rectangle, normalizing this ratio for easier regression calculation. The aspect ratio γ used to determine whether the human target recognition result meets the standard is related to the aspect ratio of the anchor box. If the aspect ratio of the predicted bounding box is similar to that of the ground truth bounding box, a smaller γ value indicates a better prediction effect. The arctan function is used to limit the value between 0 and 1 for easier regression calculation. A Ciou loss function is constructed. Loss The aspect ratio γ function detects and limits the human target recognition and localization effect of the original human target motion image after processing steps 1)-3) from two aspects: the proximity of the center point of the real box and the predicted box and the proximity of the aspect ratio. This effectively improves the accuracy and precision of human target recognition.

[0115] This invention also provides a human target recognition system, such as Figure 7 As shown, it includes a human target motion image acquisition module, a human target image preprocessing module, a human target image multi-angle fusion module, and a human target recognition and control module;

[0116] The human target motion image acquisition module is used to acquire human target motion videos in real time and input each frame of the original human target motion image from the acquired human target motion video into the model.

[0117] The human target motion image preprocessing module is used to preprocess the original human target motion images;

[0118] The multi-angle fusion module for human target images is used to perform multi-scale fusion on preprocessed human target motion images to further improve the diversity and robustness of features.

[0119] The human target recognition control module is used to construct the Ciou loss function. Loss The aspect ratio γ function is used to evaluate whether the prediction results of each point in the predicted bounding box image after multi-scale fusion meet the standard, thereby controlling whether to output the recognized human target image; if Ciou Loss If γ < 0.2 and γ < 0.03, then human target recognition is completed and the recognized human target image is output. Otherwise, repeat steps 1)-3) of the above method until the output human target recognition result meets the standard.

[0120] The Ciou loss function established in this invention. Loss By incorporating the ratio of the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, and considering the aspect ratio γ function in the calculation, this was achieved. This avoids the following formula used in existing technologies employing the Giou loss function. C represents the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box, and S represents the area of ​​the minimum bounding rectangle minus the area of ​​the union of the predicted and ground truth bounding boxes. This only considers the area relationship between the predicted and ground truth bounding boxes, without reflecting the positional relationship between them or the aspect ratio between the target boxes. This leads to... Figure 8 The results shown indicate that Giou struggles to address the issue of large differences in the aspect ratio of the predicted bounding boxes. Figure 8 As shown in the image, the bounding box that encloses the entire human body is the target bounding box, and the smaller bounding box is the predicted bounding box. The three images show that the positions of the predicted bounding boxes are different, but they are all inside the target bounding box. In all three cases, the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box is the same size, causing the C and S parameters in the Giou loss function to remain unchanged, thus degenerating the Giou loss function into the Iou loss function. Furthermore, the three images show that the aspect ratios of the predicted bounding boxes are different, resulting in significant differences in prediction performance. Because Giou does not consider aspect ratio, it struggles to handle problems with large differences in the aspect ratios of the predicted bounding boxes.

[0121] Test Example 1

[0122] To verify the effectiveness of the method of this invention, an experimental analysis was conducted on a sample of indoor walking people. The dataset for this experiment consisted of images from a public indoor people dataset and data collected online. A total of 800 images were selected and preprocessed. To reflect real-world scenarios, most of the selected images were of single or multiple people. The samples were divided into training and validation sets in a 9:1 ratio. For the rigor and professionalism of the experiment, the VOC dataset was converted into the internationally recognized COCO dataset, and the information was stored in a txt file.

[0123] The training platform for this experiment uses an NVIDIA GTX1660TI GPU and an Intel Core™ i7-9750H CPU processor. The operating system is Linux Ubuntu 16.04, and a common environment of OpenCV 4.4 and CUDA 10.2 is set up. The deep network framework used is the PyTorch framework.

[0124] To demonstrate the superiority of the improved Yolov5s model during the training phase, transfer learning was used to pre-train the model for 100 epochs on 20 classes in the VOC2007 dataset. The training results are shown below. Figure 9 As shown in the figure (Loss variation curve after 100 pre-training cycles), it can be seen from the figure that the overall Loss function converges rapidly in the first 50 epochs, and then drops to below 0.04 after 100 epochs, which has the characteristics of rapid convergence and good convergence effect.

[0125] Because this experiment involved pre-trained weights, and given the time cost of long-term training, a total of 300 epochs were performed. The training parameters were as follows: input images were uniformly 640×640 pixels; initial learning rate was 0.01; epoch learning rate was 0.2; and batch size was 16.

[0126] Test Example 2

[0127] Furthermore, the target detection evaluation metrics, such as precision and recall, of the human target recognition method provided by this invention were measured. The target detection evaluation metrics are generally precision and recall, and the calculation formulas are as follows:

[0128]

[0129]

[0130] In the formula, TP represents the number of targets identified and the result is true, FP represents the number of targets identified and the result is false, and FN represents the number of targets not identified and the result is false.

[0131] As can be seen from the two formulas above, precision P reflects whether the classifier's classification result is accurate, while recall R reflects whether the classifier can classify the target well. One emphasizes the accuracy of the result, and the other evaluates the performance of the classifier. The two evaluation metrics complement each other.

[0132] Generally, the higher the precision (P), the lower the recall (R), and vice versa. For specific details, see below. Figures 3-11 As shown, a single reference precision (P) and recall (R) are insufficient to balance the performance of a classifier. This invention uses the average precision (AP) evaluation metric as the evaluation metric for precision and recall, i.e., the area under the PR curve as the target evaluation metric. Because the PR relationship curve has a good balance point with a full convex starting point, calculating the area under the PR curve is a better metric.

[0133] The subjects of this experiment were the occupant area in a home indoor environment. The models used in the comparative experiments were the improved Yolov5s model, the Yolov5s model, and the Yolov5m model.

[0134] like Figure 10As shown in the figure, the blue line represents the iterative curve of the Yolov5s training loss function, the orange line represents the iterative curve of the Yolov5m training loss function, and the gray line represents the iterative curve of the improved Yolov5s training loss function. It is clear from the figure that the training loss function curves of the Yolov5s and improved Yolov5s models are roughly similar. Both exhibit superior performance with fast convergence in the first 15 epochs, and converge to around 0.1 after 300 epochs. Yolov5m, due to its more convolutional structures, is generally superior in the training phase, exhibiting faster convergence and a lower convergence value. The improved Yolov5s in this paper uses its own pre-trained weights. During the training phase, the iterative optimization performance of the loss function is similar to that of Yolov5s using the official pre-trained weights. The improved Yolov5s offers superior performance in terms of ease of training and shorter training time.

[0135] like Figure 11 As shown in the figure, there are two evaluation metric curves. The blue line represents the iterative curve of precision (P), and the orange line represents the iterative curve of recall (R). The figure shows that precision (P) reaches 0.9 after approximately 10 iterations and converges to around 0.9 at a very rapid iteration rate. Recall (R) also exhibits a very fast convergence rate, with the R value gradually increasing and generally remaining around 0.88 after approximately 150 iterations. Overall, both metric curves demonstrate excellent performance with rapid convergence and high values, indicating that the improved Yolov5s has fast learning capabilities and high detection accuracy.

[0136] First, the superiority of the YOLOv5 algorithm model is demonstrated by comparing it with the currently popular "one-stage" algorithm SSD and the "two-stage" Faster-R-CNN object detection algorithm. Specific performance data is shown in Table 3-2. Table 3-2 shows that the Faster-R-CNN algorithm has the highest precision and recall, detecting almost all targets. However, the image detection time is as high as half a second, which is insufficient for real-time detection in video environments. Furthermore, the number of parameters in this algorithm is approximately 20 times that of the YOLOv5s algorithm and 8 times that of the YOLOv5m algorithm, making its large computational load difficult to deploy on intelligent chips with limited computing power. The SSD algorithm, also belonging to the "one-stage" category, clearly shows low precision and recall, with values ​​approximately 10 points lower than the YOLOv5 algorithm, significantly reducing its accuracy performance. The computational speed of the SSD algorithm is between that of the YOLOv5s and YOLOv5m algorithms. Comparative analysis revealed that the YOLOvs model boasts superior performance with high detection accuracy and low computational cost. The YOLOv5s model, in particular, is highly favored by the current intelligent chip market, which has limited computing power.

[0137] To clearly demonstrate the superiority of the improved Yolov5s network model, it will be compared with the unimproved Yolov5 series Yolov5s and Yolov5m. The specific results are shown in Table 1-1. This invention makes a comprehensive evaluation using five evaluation indicators: precision P, recall R, computation time, number of parameters, and GPU floating-point computing performance. Overall, YOLOv5m outperforms YOLOv5s and the improved YOLOv5s model significantly due to its deeper network and better feature extraction capabilities. However, this translates to more network parameters and increased computation time; YOLOv5m has nearly three times the number of parameters as YOLOv5s, and its processing time for a single image is twice that of the other two models. The improved YOLOv5s model successfully reduces the number of parameters by 30% and the processing time by 0.011 seconds compared to the YOLOv5s model. In terms of precision (P), the improved YOLOv5s outperforms the YOLOv5s model by 0.6%, and in terms of recall (R), it outperforms the YOLOv5s model by 1.24%, with its performance approaching that of YOLOv5m in both metrics. In terms of GPU computing performance, the improved YOLOv5s requires only 2 / 3 of the GPU performance of YOLOv5 and only 1 / 5 of the performance of YOLOv5m.

[0138] In summary, the improved human target recognition method provided in this invention successfully enhances the performance of YOLOv5s, achieving the original detection accuracy with fewer parameters and lower GPU computing power, and its performance approaches that of the larger YOLOv5m model. To further demonstrate the superiority of this improvement, an improved YOLOv5m model was added. Since larger YOLOv5 models involve increasing the number of residuals, the improved method described in this paper is used more extensively. The computational cost of the m (medium) level model has been reduced by half in terms of parameter count and GPU floating-point performance, making it more suitable for low-end GPU chip products.

[0139] Table 1-1 Performance comparison of the three algorithms

[0140]

[0141]

[0142] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0144] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A human target recognition method, an improved lightweight Yolov5s network target detection model is used, characterized in that, The method comprises the following steps: 1) collecting a human target motion video in real time, each frame of original human target motion picture of the human target motion video being input into a model; 2) preprocessing the original human target motion picture; 3) performing multi-scale fusion on the human target motion picture after preprocessing obtained in the step 2); 4) Construct Ciou loss function and aspect ratio function; 5) evaluate the human target recognition positioning loss function of the multi-scale fusion after step 3) the predicted frame image in the frame and the real frame image Value and width-height ratio Function value, if And Complete human target recognition, and output the recognized human target image, otherwise repeat steps 1) -3). The preprocessing of the original human target motion picture in the step 2) comprises: 2.1) reducing the dimension of the original human target motion picture of size X X 3 collected in step 1) to obtain a feature map of size , and saving the information on the original human target motion picture on the channel number; 2.2) performing two-group convolution on the picture obtained in the step 2.1), and splicing to obtain a final feature map of a backbone; 2.3) performing depth separable convolution and group convolution with g groups on the final feature map spliced in the step 2.2), and reducing the number of parameters of the overall network through a multiplication decomposition step; 2.4) repeating the steps 2.2)-2.3) twice to complete the preprocessing of the original human target motion picture; The multi-scale fusion on the human target motion picture after preprocessing obtained in the step 2) in the step 3) comprises the following steps: 3.1) performing two-group convolution on the human target motion picture after preprocessing, and splicing to obtain a final feature map of Neck feature processing; 3.2) using different sizes of pooling windows to respectively pool data, then splicing the pooling results of different pooling windows for multi-scale fusion; 3.3) performing depth separable convolution and group convolution with g groups on the image after multi-scale fusion in the step 3.2), and reducing the number of parameters of the overall network through a multiplication decomposition step; 3.4) simultaneously using mean pooling and maximum pooling on the picture obtained after the step 3.3), constructing a CBAM convolution attention pooling model, and pooling the input feature map with a size of HxWxC after the step 3.3) into an output feature map with a size of HxWx1; 3.5) outputting after ordinary convolution on the output feature map with a size of HxWx1 obtained in the step 3.4); The CBAM convolution attention pooling model additionally introduces a spatial attention mechanism compared with the compression and excitation network SElayer, and is composed of two steps, which are respectively a channel attention mechanism and a spatial attention mechanism; the channel attention mechanism is intended for weighting of pixel values on each feature map, pooling each feature map, and focusing on one-dimensional data weighting; and the spatial attention mechanism turns to learning and optimization on two-dimensional positions, and simultaneously uses mean pooling and maximum pooling to pool the input feature map with a size of HxWxC into an output feature map with a size of HxWx1; The spatial attention mechanism fuses the two one-dimensional feature maps obtained after processing of the original feature map into a two-dimensional feature map, generates a feature map with a channel number of one dimension through convolution operation, and finally weights into the original feature map to realize optimization on two-dimensional positions.

2. The human target recognition method of claim 1, wherein, The size of X in the step 2.1) is 80-640.

3. The human target recognition method of claim 1, wherein, The CBAM convolution attention pooling model constructed in the step 3.4) has the following formula: ； wherein, denotes a sigmoid activation function, denotes a convolution operation with a 7x7 kernel, denotes mean pooling, denotes max pooling, and F is the input feature map of size HxWxC after the step 3.3).

4. The human target recognition method of claim 1, wherein, The size of the pooling window in the step 3.2) is 5x5, 9x9 and 13x13.

5. The human target recognition method of claim 1, wherein, The deep separable convolution comprises: converting a feature map with an input size of × N standard convolution kernels × M, and then splitting each of the standard convolution kernels into M × 1 deep convolution and 1x1xM point convolution, and the convolution calculation formula is as follows: ； wherein, represents the deep convolution calculation amount, represents the point-by-point convolution calculation amount, represents the convolution kernel size, M is the number of channels of the input feature map, and N is the number of channels of the output feature layer.

6. The human target recognition method of claim 1, wherein, The g-grouped group convolution is to divide the input feature map with the dimension of HxWxC1 into g groups according to the channel number, and then obtain a grouped input feature map with the dimension of HxWxCg. the dimension of HxWxCg. The convolution kernel size of the input feature map with the dimension of HxWxC1 is KxKxC1 , the dimension of the grouped input feature map is HxWxC2 , and the dimension of the spliced final feature map is HxWxC2, that is, the dimension of the backbone final feature map spliced by the step 2.2) and the step 3.1) and the dimension of the Neck feature processing final feature map are both HxWxC2; wherein H is the height of the input feature map, W is the width of the input feature map, C1 is the channel number of the input feature map, C2 is the channel number of the final feature map, and K is the convolution kernel length value and width value of the grouped convolution. The calculation formula of the grouped convolution is .

7. The human target recognition method of claim 1, wherein, The Ciou loss function in the step 4) As follows: ； wherein, represents the Euclidean distance between the two center points of the predicted box and the real box, represents the diagonal distance of the minimum circumscribed rectangle of the predicted box and the real box; is the width-height ratio, is the intersection ratio of the predicted box and the real box, ; The aspect ratio The function is as follows: ； wherein, represents a width of the real box, represents a height of the real box, represents a width of the predicted box, represents a height of the predicted box.

8. A human target recognition system for performing the human target recognition method according to claim 1, characterized by The human target motion picture acquisition module, the human target picture preprocessing module, the human target picture multi-angle fusion module and the human target recognition control module are included. The human target motion picture acquisition module is used for acquiring human target motion video in real time, and inputting each frame of original human target motion picture in the acquired human target motion video into a model. The human target picture preprocessing module is used for preprocessing the original human target motion picture. The human target picture multi-angle fusion module is used for performing multi-scale fusion on the human target motion picture after preprocessing. The human body target recognition control module is configured to construct a Ciou loss function And the width-height ratio The function evaluates whether the prediction effect of each point in the image in the prediction frame after multi-scale fusion and each point in the real frame conforms to the standard, and further controls whether to output the recognized human body target image.

Images