Personnel detection method and device based on top-mounted camera, and program product

Abstract

The invention provides a personnel detection method, equipment and a program product, personnel are detected based on a top-mounted camera, and the method comprises the following steps: a target detection network structure receives a plurality of continuous video frames; for each frame, the target detection network structure identifies whether the target person has a predetermined label pattern in a predetermined body area, and counts the total number of frames having the predetermined label pattern; if the total frame number is greater than a preset number threshold value, determining that the target person is a person meeting a preset attribute; otherwise, determining that the target person is a person which does not meet the predetermined attribute; the number threshold is less than a predetermined number and greater than or equal to 2; wherein the target detection network structure is an improved network structure based on YOLO-V5; wherein the YOLOv8 network structure C2f is used for replacing C3 in the YOLO-V5 network structure, the SSPF of the YOLO-V5 network structure is improved into the SSPFM, and the CBAM module of the YOLO-V5 network structure is improved into the CBAMLSA. By means of the technical scheme, the personnel meeting the preset attributes can be accurately recognized in the scene of the top-mounted camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target recognition, and in particular to a personnel detection method, device and program product based on a top-mounted camera. BACKGROUND

[0002] At present, existing personnel detection, such as company employee detection, is mostly based on facial features for detection. However, in a top-mounted camera scene, the camera cannot capture the face of a person, but only the head and shoulder of the person; therefore, a scheme for recognizing personnel attributes, such as recognizing whether a person is an employee, by facial feature detection is not feasible. SUMMARY

[0003] Embodiments of the present application provide a personnel detection method, device and program product to achieve more accurate personnel attribute recognition in a top-mounted camera scene.

[0004] To achieve the above purpose, on the one hand, a personnel detection method is provided, which detects personnel based on a top-mounted camera, comprising:

[0005] A pre-trained target detection network structure receives a predetermined number of consecutive video frames, the predetermined number being greater than 2;

[0006] For each frame of the predetermined number of consecutive video frames, the pre-trained target detection network structure identifies whether a target person in the image of the video frame has a predetermined label pattern in a predetermined body region, and counts the total number of frames having the predetermined label pattern; wherein the predetermined body region is a pre-set body region that can be captured by the top-mounted camera;

[0007] If the total number of frames having the predetermined label pattern is greater than a predetermined number threshold, the target person is determined to be a person satisfying a predetermined attribute; otherwise, the target person is determined to be a person not satisfying the predetermined attribute; the predetermined number threshold is less than the predetermined number and greater than or equal to 2;

[0008] Wherein, the step of identifying whether a target person in the image of the video frame has a predetermined label pattern in a predetermined body region includes the following steps:

[0009] The image of the video frame is subjected to a first ConvBNRelu6 operator to output a first feature map P1;

[0010] The first feature map is subjected to a second ConvBNRelu6 operator to output a second feature map P2;

[0011] The second feature map P2 is sequentially subjected to a first C2f operator, a third ConvBNRelu6 operator and a second C2f operator to output a third feature map P3;

[0012] The third feature map P3 sequentially passes through a fourth ConvBNRelu6 operator, a third C2f operator, an SPPFM operator and a fifth ConvBNRelu6 layer, and outputs a fourth feature map N2;

[0013] N2 = ConvBNRelu6 (SPPFM (C2f (ConvBNRelu6 (P3))))

[0014] The SPPFM operator includes a MaxPool2d module and an AvgPool2d module, and the feature map output by the SPPFM operator is a feature map obtained by adding the feature map output by the MaxPool2d module and the feature map output by the AvgPool2d module in the channel dimension, and then performing a ConvBNRelu6 operator operation;

[0015] The fourth feature map N2 is up-sampled by an UpSample operator, and is concatenated with the third feature map P3 by a first Concat operator, and the output of the first Concat operator sequentially passes through a CBAM-LSA operator and a fourth C2f operator, and outputs a fifth feature map N1;

[0016] N1 = C2f (CBAM_LSA (Concat (UpSample (N2), P3)))

[0017] The CBAM-LSA operator includes a CBAM_LSA channel attention module and a CBAM_LSA spatial attention module, wherein:

[0018] y1 = Clamp (alpha * Conv2d (AvgPool2d (x)) + beta)

[0019] y2 = Clamp (alpha * Conv2d (y1) + beta)

[0020] y1 is the output of the CBAM_LSA channel attention module, y2 is the output of the CBAM_LSA spatial attention module, alpha is a constant 0.1, beta is a constant 0.5, and Clamp is a numerical truncation operator, which sets the output value to 0 when the output value is less than 0, or sets the output value to 1 when the output value is greater than 1;

[0021] The fifth feature map N1 sequentially passes through a first Conv2d operator and a first Sigmoid operator to output a sixth feature map O1;

[0022] O1 = Sigmoid (Conv2d (N1))

[0023] wherein the fifth feature map N1 is copied, then passes through a sixth ConvBNRelu6 operator, and then passes through a second Concat operator, a second CBAM-LSA operator and a fifth C2f operator with the fourth feature map N2 in sequence, to output a seventh feature map N3;

[0024] N3=C2f(CBAM_LSA(Concat(ConvBNRelu6(N1),N2)))

[0025] The seventh feature map N3 passes through a second Conv2d operator and a second Sigmoid layer in sequence, to output an eighth feature map O2;

[0026] O2=Sigmoid(Conv2d(N3))

[0027] The sixth feature map O1 and the eighth feature map O2 are merged, the position of the detection frame corresponding to the predetermined region of the body is extracted from the merged feature map, and the confidence that the predetermined label pattern exists in the detection frame is obtained.

[0028] When the confidence is greater than a predetermined confidence threshold, it is determined that the predetermined label image exists in the frame.

[0029] Preferably, in the personnel detection method, the calculation process of the SPPFM operator comprises:

[0030] M1=MaxPool2d(x)

[0031] M2=MaxPool2d(M1)

[0032] M3=MaxPool2d(M2)

[0033] MS=Concat(M1,M2,M3)

[0034] A1=AvgPool2d(x)

[0035] A2=AvgPool2d(A1)

[0036] A3=AvgPool2d(A2)

[0037] AS=Concat(A1,A2,A3)

[0038] MAS=Add(MS,AS)

[0039] y=ConvBNRelu6(MAS)

[0040] Wherein, x represents an input feature map; M1, M2, M3, MS, A1, A2, A3, AS and MAS represent output intermediate feature maps; Add represents an operator of adding two feature maps in a channel dimension; and y is a feature map output by the SPPFM operator.

[0041] Preferably, the personnel detection method obtains the training samples for training the target detection network structure through the following steps:

[0042] The difficult negative samples for target detection are pre-set, and the difficult negative samples include: a pattern displayed on a mobile phone screen in an environment, a pattern on a chest of a human body wearing clothes, and / or a pattern of a surrounding environment.

[0043] Randomly sampling in the difficult negative samples and positive samples, and splicing at least one positive sample and at least one difficult negative sample to form a training sample set.

[0044] Preferably, the personnel detection method splices the difficult negative samples and the positive samples according to a predetermined quantity ratio.

[0045] Preferably, the personnel detection method has the following technical effects:

[0046] Preferably, the personnel detection method further comprises one or more of the following:

[0047] The predetermined label pattern is a pattern of black and white stripes or red and white stripes.

[0048] The predetermined body area includes a predetermined shoulder area and / or a predetermined chest area.

[0049] Preferably, the personnel detection method has the following technical effects:

[0050] Preferably, the personnel detection method identifies whether the target personnel has the predetermined label pattern in the predetermined body area by detecting whether a pattern of a shoulder badge and / or a chest badge of the target personnel is the predetermined label pattern.

[0051] The above technical solutions have the following technical effects:

[0052] The technical scheme of the embodiment of the present application improves the existing traditional YOLO-V5 network structure, uses a YOLOv8 network structure C2f to replace the C3 module in the YOLO-V5 network structure, improves the SSPF of the YOLO-V5 network into SSPFM, and improves the CBAM module of the YOLO-V5 network structure into CBAM_LSA, so that in the conventional scene of the top-mounted camera, the problem of unstable features caused by the angle change of the chest badge and the shoulder badge due to the movement of the human body can be solved, and the preset label of the personnel attribute can be accurately identified;

[0053] Further, the embodiment of the present application uses black and white stripes and red and white stripes with high imaging contrast in the conventional constant scene of the camera as specific predetermined label patterns, thereby improving the accuracy of personnel detection;

[0054] Further, in the training process of the target detection network structure, the embodiment of the present application adds new samples formed by splicing difficult negative samples and positive samples to the training sample set, so that in the scene of the top-mounted camera, the problem of similar patterns being mixed in the detection of the patterns of the chest badge and the shoulder badge, which leads to the similar patterns being incorrectly detected as the label of the new employee, i.e., the existence of the predetermined pattern, can be avoided, and the accuracy of personnel detection is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The flowchart of the personnel detection method of an embodiment of the present application;

[0056] Figure 2 The flowchart of the target detection network structure identifying whether the target personnel in the image of the video frame exists the predetermined label pattern in the predetermined body area in an embodiment of the present application;

[0057] Figure 3 And Figure 4 Respectively, the pattern of black and white stripes or red and white stripes;

[0058] Figure 5 The detection process diagram of the personnel detection method of another embodiment of the present application;

[0059] Figure 6 The structure diagram of the SSPF operator;

[0060] Figure 7 The structure diagram of the SSPFM operator;

[0061] Figure 8 The local 6*6 feature map;

[0062] Figure 9This is a schematic diagram of kernel size = 5*5;

[0063] Figure 10 It is the feature map of the final output of MaxPool2d;

[0064] Figure 11 It is the feature map of the final output of AvgPool2d;

[0065] Figure 12 It is a schematic diagram of the existing CBAM module structure;

[0066] Figure 13 It is a structural diagram of the existing CBAM_LSA module;

[0067] Figure 14 This is an example of a 224*448 resolution image;

[0068] Figure 15 This is an example of a 448*224 resolution image;

[0069] Figure 16 This is an example of a 448*448 resolution image. DETAILED DESCRIPTION

[0070] To further illustrate various embodiments, the present invention is provided with accompanying drawings. These drawings form part of the present disclosure and are primarily used to illustrate the embodiments and, in conjunction with the relevant description in the specification, to explain the operating principles of the embodiments. By referring to these drawings, one of ordinary skill in the art will understand other possible embodiments and the advantages of the present invention. The components in the figures are not drawn to scale, and similar reference numerals are generally used to represent similar components.

[0071] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0072] Example 1:

[0073] Figure 1 FIG. 4 is a flow chart of a person detection method according to an embodiment of the present invention. Figure 2 This is a flow chart of a target detection network structure used in a person detection method according to an embodiment to identify whether a target person in an image of a video frame has a predetermined label pattern in a predetermined area of ​​the body. Figure 1 and Figure 2 The personnel detection method based on the top-mounted camera according to an embodiment of the present invention is described.

[0074] like Figure 1 , a person detection method according to an embodiment of the present invention includes:

[0075] S1, the pre-trained target detection network structure receives a predetermined number of continuous video frames, the predetermined number being greater than 2;

[0076] S2, for each frame in the predetermined number of continuous video frames, the pre-trained target detection network structure identifies whether the target person in the image of the video frame has a predetermined label pattern in a predetermined body region, and counts the total number of frames with the predetermined label pattern; wherein the predetermined body region is a body region that can be captured by the top-mounted camera and is pre-set;

[0077] S3, if the total number of frames with the predetermined label pattern is greater than a predetermined number threshold, the target person is determined to be a person meeting the predetermined attribute; otherwise, the target person is determined to be a person not meeting the predetermined attribute; the predetermined number threshold is less than the predetermined number and greater than or equal to 2;

[0078] Wherein, as Figure 2 the target detection network structure identifies whether the target person in the image of the video frame has a predetermined label pattern in a predetermined body region includes the following steps:

[0079] S201, the image of the video frame is subjected to a first ConvBNRelu6 operator to output a first feature map P1;

[0080] S202, the first feature map is subjected to a second ConvBNRelu6 operator to output a second feature map P2;

[0081] S203, the second feature map P2 is sequentially subjected to a first C2f operator, a third ConvBNRelu6 operator and a second C2f operator to output a third feature map P3;

[0082] S204, the third feature map P3 is sequentially subjected to a fourth ConvBNRelu6 operator, a third C2f operator, an SPPFM operator and a fifth ConvBNRelu6 layer to output a fourth feature map N2;

[0083] N2 = ConvBNRelu6 (SPPFM (C2f (ConvBNRelu6 (P3))))

[0084] Wherein, the SPPFM operator includes a MaxPool2d module and an AvgPool2d module, and the feature map output by the SPPFM operator is: the feature map output by the MaxPool2d module and the feature map output by the AvgPool2d module are added in the channel dimension, and then subjected to a ConvBNRelu6 operator operation;

[0085] S205, the fourth feature map N2 is up-sampled by an UpSample operator, and is concatenated with the third feature map P3 by a first Concat operator, and the output of the first Concat operator is sequentially subjected to a CBAM-LSA operator and a fourth C2f operator, to output a fifth feature map N1;

[0086] N1 = C2f(CBAM_LSA(Concat(UpSample(N2), P3)))

[0087] The CBAM-LSA operator includes a CBAM_LSA channel attention module and a CBAM_LSA spatial attention module, wherein:

[0088] y1 = Clamp(alpha * Conv2d(AvgPool2d(x)) + beta)

[0089] y2 = Clamp(alpha * Conv2d(y1) + beta)

[0090] wherein y1 is the output of the CBAM_LSA channel attention module, y2 is the output of the CBAM_LSA spatial attention module, alpha is a constant 0.1, beta is a constant 0.5, and Clamp is a numerical clipping operator that sets the output value to 0 when the output value is less than 0 or sets the output value to 1 when the output value is greater than 1;

[0091] S206, the fifth feature map N1 is sequentially subjected to a first Conv2d operator and a first Sigmoid operator to output a sixth feature map O1;

[0092] O1 = Sigmoid(Conv2d(N1))

[0093] S207, after the fifth feature map N1 is copied, it is subjected to a sixth ConvBNRelu6 operator, and is concatenated with the fourth feature map N2 by a second Concat operator, a second CBAM-LSA operator, and a fifth C2f operator, to output a seventh feature map N3;

[0094] N3 = C2f(CBAM_LSA(Concat(ConvBNReLu6(N1), N2)))

[0095] S208, the seventh feature map N3 is sequentially subjected to a second Conv2d operator and a second Sigmoid layer to output an eighth feature map O2;

[0096] O2 = Sigmoid(Conv2d(N3))

[0097] S209, merge the sixth feature map O1 and the eighth feature map O2, extract the position of the detection box corresponding to the predetermined body region from the merged feature map, and obtain the confidence that the predetermined label pattern exists in the detection box;

[0098] S210, when the confidence is greater than a predetermined confidence threshold, it is determined that the predetermined label image exists in the frame.

[0099] Embodiment two:

[0100] In this embodiment of the application, the personnel detection method based on the top-mounted camera is applied to detecting and identifying new employees in an indoor environment; the predetermined body region is the shoulder or the chest; whether the target personnel has the predetermined label pattern in the predetermined body region is identified by detecting whether the pattern of the shoulder badge and / or the chest badge of the target personnel is the predetermined label pattern. In this embodiment, preferably, the predetermined label pattern adopts a black-and-white stripe pattern or a red-and-white stripe pattern. Figure 3 and Figure 4 The black-and-white stripe pattern or the red-and-white stripe pattern is exemplary. In other implementations, those skilled in the art know that the black-and-white stripe pattern or the red-and-white stripe pattern can have many different implementations, which are not described here. In a conventional constant bright scene, the imaging contrast of the black-and-white stripe and the red-and-white stripe is high, which is easier for the detection algorithm of the target detection network structure to detect and identify. In other embodiments, the personnel detection method based on the top-mounted camera can be applied to the detection of other personnel attributes.

[0101] The target detection network structure used in the personnel detection method of the embodiment of the application is based on the conventional YOLO-V5 network structure as the basis of the detection model, and is improved on this basis to obtain; Figure 5 Fig. 1 is a schematic diagram of the detection process of the personnel detection method of this embodiment of the application; the following will be described in combination with Figure 5 From the input side, model detection and output side of the target detection network structure, the detection process of the personnel by the target detection network structure is described, which is the detection process of the new employee in this example.

[0102] The input side: the model input is a continuous 5-frame rate RGB-3 channel image, and the image resolution is 224*244; the continuous 5-frame rate and the image resolution here are exemplary, and other frame rates and image resolutions can be used in other implementations.

[0103] Model detection: the input side of the continuous 5 frame rate image will pass through the network architecture inference process, which is composed of multiple network layers, including ConvBNRelu6, C2f, SPPFM, UpSample, CBAM_LSA, Concat, Conv2d and Sigmoid. Among them, ConvBNRelu6 is composed of 3*3 convolution kernel convolution layer Conv2d, batch normalization layer BatchNorm2d and activation layer ReLU6; C2f is composed of 1*1 convolution kernel convolution layer Conv2d in front of 1 times width convolution layer, multiple 3*3 convolution kernel convolution layer Conv2d, 1.5 times width convolution layer 1*1 convolution kernel convolution layer Conv2d; SPPFM is a spatial pyramid pooling layer; UpSample is a 2 times up sampling layer, and the up sampling method is nearest neighbor; CBAM_LSA is a linear spatial attention mechanism module; Concat is an operator that merges two input feature maps in channel dimension; Sigmoid is a normalized activation function.

[0104] Firstly, the input image will pass through two layers of ConvBNRelu6, and then pass through a layer of C2f operator, a layer of ConvBNRelu6 operator and a layer of C2f operator to output P3 feature map;

[0105] P1 = ConvBNRelu6(x)

[0106] P2 = ConvBNRelu6(P1)

[0107] P3 = C2f(ConvBNRelu6(C2f(P2)))

[0108] Among them, P1 and P2 represent the intermediate output feature map, and P3 represents the final output feature map; P1 resolution is 112*112, and channel number is 24; P2 resolution is 56*56, and channel number is 48; P3 resolution is 28*28, and channel number is 96.

[0109] Secondly, P3 feature map will pass through a layer of ConvBNRelu6 operator, a layer of C2f operator, a layer of SPPFM operator and a layer of ConvBNRelu6 operator to output N2 feature map;

[0110] N2 = ConvBNRelu6(SPPFM(C2f(ConvBNRelu6(P3))))

[0111] N2 feature map will pass through UpSample operator for up sampling, and will be operated by Concat operator with output P3 feature map, and then output N1 feature map through CBAM-LSA module and C2f operator;

[0112] N1 = C2f(CBAM_LSA(Concat(UpSample(N2), P3)))

[0113] N1 feature map is further processed by a Conv2d layer and a Sigmoid layer to output O1 feature map:

[0114] O1 = Sigmoid(Conv2d(N1))

[0115] Meanwhile, N1 feature map is copied and processed by a ConvBNRelu6 operator and combined with N2 feature map Figure 1 through a Concat operator, CBAM-LSA, C2f operator, and then processed by a Conv2d and Sigmoid layer to output O2 feature map.

[0116] N3 = C2f(CBAM_LSA(Concat(ConvBNRelu6(N1), N2)))

[0117] O2 = Sigmoid(Conv2d(N3))

[0118] The outputs of O1 feature map and O2 feature map are combined, and the positions xywhs and scores of the epaulettes and breast badges detection boxes are extracted therefrom; xywhs represents the horizontal coordinate, vertical coordinate, width, and height of the center point of the breast badge or epaulettes detection box; and scores represents a list of confidence scores of the breast badge or epaulettes in the detection box.

[0119] Output side: The output side mainly describes the determination process of the new employee. Since the value range of scores is [0, 1], preferably, half of the probability value, i.e., 0.5, is selected to determine whether there is a breast badge or epaulettes in each frame rate image. When the value in scores is greater than 0.5, the model considers that the frame rate has a shoulder badge or chest badge label; on the contrary, if the value in scores is less than 0.5, the model considers that the frame rate does not have a shoulder badge or chest badge label. Assuming that the scores of 5 consecutive frame rates are [0.71, 0.65, 0.25, 0.82, 0.37], the 1st frame rate image, the 2nd frame rate image, and the 4th frame rate image are marked as “staff”, and the 3rd frame rate image and the 5th frame rate image are marked as “no_staff”, and thus the output label list is [staff, staff, no_staff, staff, no_staff]. The number of “staff” appearing in the label list is counted, and if the number exceeds a predetermined number threshold, such as 2, it is determined that the person is an employee, otherwise it is determined that the person is not an employee. In other implementations, the number threshold can be set to other values.

[0120] For example, Figure 5The target detection network structure adopted in the embodiment of the present application improves the existing conventional YOLO-V5 network structure as follows.

[0121] The conventional YOLO-V5 network structure has sizes including small, middle and large, and these model structures have some redundancy, including too many channels, too many network layers and too dense neuron quantity, which is somewhat a waste of resources and computing power in the small image detection task of shoulder badges and chest cards. Therefore, in the embodiment of the present application, the YOLO-V5 network structure is designed in detail, the C2f of the YOLOv8 network structure is used to replace the C3 module in the YOLO-V5 network structure, the SSPF of the YOLO-V5 network is improved into SSPFM, and the CBAM module of the YOLO-V5 network structure is improved into CBAM_LSA. The structure of the SSPF is as follows Figure 6 , and the structure of the SSPFM is as follows Figure 7 .

[0122] The following specifically describes the improvement of the SSPFM used in the present application to the SSPF of the existing YOLO-V5 network.

[0123] The existing conventional YOLOV5 adds a MaxPool2d module with kernel size = 5*5, stride = 1*1 and pooling = 2*2 in the network SPPF layer, and the purpose is to fuse large target features and small target features to improve the model detection rate; however, MaxPool2d has a disadvantage, that is, the module output is too sparse, because it only retains the maximum value in the 5*5 grid as the feature output, while the AvgPool2d module outputs the average value in the 5*5 grid, which can make up for the disadvantage of MaxPool2d that may lose local features, refine the granularity of model detection, and improve the recall rate of the model in detecting chest cards and badges.

[0124] The original SSPF operator process is represented as

[0125] M1 = MaxPool2d(x)

[0126] M2 = MaxPool2d(M1)

[0127] M3 = MaxPool2d(M2)

[0128] MS = Concat(M1, M2, M3)

[0129] y = ConvBNRelu6(MS)

[0130] Wherein, x represents the input feature map, M1, M2, M3 and MS all represent intermediate feature map output results, and y is the final feature map output; MaxPool2d represents a maximum pooling downsampling operator.

[0131] The improved SPPFM operator process adopted by the embodiment of the application is represented as

[0132] M1=MaxPool2d(x)

[0133] M2=MaxPool2d(M1)

[0134] M3=MaxPool2d(M2)

[0135] MS=Concat(M1,M2,M3)

[0136] A1=AvgPool2d(x)

[0137] A2=AvgPool2d(A1)

[0138] A3=AvgPool2d(A2)

[0139] AS=Concat(A1,A2,A3)

[0140] MAS=Add(MS,AS)

[0141] y=ConvBNRelu6(MAS)

[0142] Wherein, A1, A2, A3, AS and MAS all represent intermediate feature map output results; AvgPool2d represents an average pooling downsampling operator, and Add represents an operator of adding two feature maps in the channel dimension.

[0143] The detailed process of MaxPool2d and AvgPool2d operations will be explained below in conjunction with Figures 8-11 Wherein, Figure 8 is a local 6*6 feature map; Figure 9 is a schematic diagram of kernel size=5*5; Figure 10 is the final output feature map of MaxPool2d; Figure 11 is the final output feature map of AvgPool2d.

[0144] The MaxPool2d calculation process of kernel size=5*5, stride=1*1, pooling=2*2 is as follows:

[0145] Max_value1=Max(1*1,2*-1,3*1,...,3*0,2*0)=3

[0146] Max_value2=Max(1*2,3*-1,2*1,...,3*0,2*0)=4

[0147] Max_value3=Max(1*1,2*-1,3*1,...,3*0,2*0)=4

[0148] Max_value4=Max(1*1,2*-1,3*1,...,3*0,2*0)=3

[0149] The calculation process of AvgPool2d with kernel size = 5*5, stride = 1*1, and pooling = 2*2 is:

[0150] Avg_value1=((1*1)+(2*-1)+...+(2*0)) / 25=3 / 25=0.12

[0151] Avg_value2=((1*1)+(2*-1)+...+(2*0)) / 25=11 / 25=0.44

[0152] Avg_value3=((1*1)+(2*-1)+...+(2*0)) / 25=5 / 25=0.2

[0153] Avg_value4=((1*1)+(2*-1)+...+(2*0)) / 25=4 / 25=0.16

[0154] Figure 12 It is a schematic diagram of the existing CBAM module structure; Figure 13 This is a structural diagram of the existing CBAM_LSA (CBAM_LinearSpatial Attention) module.

[0155] The following describes the improvement from the CBAM module, i.e., the CBAM operator, of the YOLO-V5 network structure to the CBAM_LSA used in the target detection network structure in an embodiment of the present invention.

[0156] The existing traditional YOLOV5 intersperses CBAM modules between network layers. The CBAM module includes the channel attention processing process and the spatial attention calculation process.

[0157] The existing original CBAM module process is expressed as:

[0158] ①CBAM-ChannelAttention

[0159] y1=x*sigmoid(Conv2d(AvgPool2d(x)))

[0160] where x denotes the input feature map, AvgPool2d still represents the average pooling down-sampling operator, Conv2d represents the convolution operator, and Sigmoid represents the normalized activation function.

[0161] ②CBAM-SpatialAttention

[0162] y1_mean=ChannelMean(y1)

[0163] y1_max=ChannelMax(y1)

[0164] y2=x*Sigmoid(Conv2d(Concat(y1_mean,y1_max)))

[0165] where ChannelMean represents an operator for calculating the mean value of a feature map along the channel dimension, and ChannelMax represents an operator for calculating the maximum value of a feature map along the channel dimension.

[0166] The CBAM_LSA module process adopted in the target detection network structure of the embodiment of the application is represented as follows:

[0167] ①CBAM_LSA-ChannelAttention

[0168] y1=Clamp(alpha*Conv2d(AvgPool2d(x))+beta)

[0169] where alpha is a constant 0.1, beta is a constant 0.5, Clamp is a numerical truncation operator, and when the output value is lower than 0, the value is set to 0, or when the output value is higher than 1, the value is set to 1.

[0170] ②CBAM_LSA-SpatialAttention

[0171] y2=Clamp(alpha*Conv2d(y1)+beta)

[0172] where alpha is still a constant 0.1, and beta is still a constant 0.5.

[0173] For the SpatialAttention calculation process, the target detection network structure of the embodiment of the application does not directly calculate the mean and maximum values of the channels on the input side, and after the Concat operator, it is sent to the Conv2d convolution layer with a kernel size of 7*7, an input channel of 2, and an output channel of 1; instead, a convolutional layer with a kernel size of 7*7, an input channel consistent with the channel of x (here x refers to the feature map output by the upstream), and an output channel of 1 is directly used; in this way, the calculation amount of the mean and maximum values is saved, and the weight of each 7*7 feature map on the feature map is balanced, so that the model learns by itself.

[0174] The CBAM_LSA module, i.e., the operator, multiplies the convolution output value by alpha = 0.1 and adds beta = 0.5 in the ChannelAttention and SpatialAttention calculation process, which adjusts the distribution range and offset of the attention weight through linear transformation, thereby optimizing the effect of the spatial attention mechanism. Multiplying by alpha prevents the original attention score from having an unstable distribution range, and this value may be too large or too small, which may cause gradient explosion or disappearance if used directly. By multiplying by alpha = 0.1, the value is reduced to a more reasonable range, avoiding numerical instability. Adding beta ensures that the attention score remains a certain positive value after scaling, controlling the degree of retention of the attention mechanism for low response areas. Finally, we use the clamp operator to constrain the value between 0.0 and 1.0. The above process can replace the Sigmoid operator to avoid the extreme nature of the Sigmoid operator in numerical mapping, i.e., leading to an uneven distribution of negative and positive extreme values.

[0175] In a specific implementation, the target detection network structure model may encounter some Negative Hard cases during the detection of shoulder badges and chest badges, which makes it difficult for the model to distinguish whether these Negative Hard cases are positive samples. Negative Hard cases include patterns displayed on mobile phone screens, patterns on the chests of people wearing clothes, and / or surrounding environments, etc. In order to enable the model to better distinguish the features of Negative Hard cases and positive samples, in further embodiments, Negative Hard cases for target detection are pre-set, and training sample sets are generated by splicing Negative Hard cases and positive samples to reduce false detection of shoulder badges and / or chest badges.

[0176] In one specific implementation, the splicing manner is: sampling randomly in Negative Hard cases and positive samples, wherein at least one hard negative sample and at least one positive sample are spliced to form a new sample. For example, then spliced into a new sample according to a predetermined quantity ratio. In one specific implementation, spliced into a new sample according to a predetermined length-width ratio or image resolution. Preferably, the hard negative sample and the positive sample are spliced into a new training sample according to 1*2, 2*1 and 2*2 cases, with resolutions of 224*448, 448*224 and 448*448 respectively, and sent to the training set for model learning. For example: the sample composition of 1*2 includes: 1 positive sample and 1 hard negative sample; the sample composition of 2*1 includes: 1 positive sample and 1 hard negative sample; the sample composition of 2*2 includes: 1 positive sample and 3 hard negative samples; 2 positive samples and 2 hard negative samples; 3 positive samples and 1 hard negative sample. In one specific implementation, the positions of the samples are random. In one specific implementation, the reason for using the length-width ratio of 1:2, 2:1 and 1:1 is that the resolution ratio of the pictures in the original data set is between 1:2-2:1. Among them, Figure 14 is an example of a 224*448 resolution image; Figure 15 is an example of a 448*224 resolution image; Figure 16 is an example of a 448*448 resolution image.

[0177] Embodiment three:

[0178] The application further provides an electronic device, which comprises a processor and a memory, and a computer program stored in the memory and executable on the processor, wherein the processor comprises one or more than one processing core, the memory is connected to the processor through a bus, the memory is used for storing program instructions, and the processor implements the steps in the above method embodiments of the embodiment one of the application when executing the computer program.

[0179] Further, as an executable scheme, the electronic device can be a computer unit, which can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer unit can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned component structure of the computer unit is only an example of the computer unit, and does not constitute a limitation on the computer unit, and can include more or fewer components than the above, or combine certain components, or different components. For example, the computer unit can also include an input / output device, a network access device, a bus and the like, and the embodiments of the application do not limit this.

[0180] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like, which is a control center of the computer unit, and connects all parts of the computer unit through various interfaces and lines.

[0181] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer unit by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; and the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0182] Embodiment four:

[0183] The application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method as described above.

[0184] Although the application is specifically shown and described in connection with the preferred embodiments, those skilled in the art should understand that various changes in form and details can be made to the application without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A method for detecting a person, characterized in that: Personnel detection based on top-mounted cameras, including: The pre-trained object detection network structure receives a predetermined number of consecutive video frames, wherein the predetermined number is greater than 2; For each of the predetermined number of consecutive video frames, the pre-trained target detection network structure identifies whether a predetermined label pattern exists in a predetermined region of the body of the target person in the image of the video frame, and counts the total number of frames in which the predetermined label pattern exists; wherein the predetermined region of the body is a predetermined region of the body that can be captured by the top-mounted camera; If the total number of frames containing the predetermined label pattern is greater than a predetermined number threshold, the target person is determined to be a person who meets the predetermined attribute; otherwise, the target person is determined to be a person who does not meet the predetermined attribute; the predetermined number threshold is less than the predetermined number and greater than or equal to 2; The target detection network structure identifies whether a target person in an image of a video frame has a predetermined label pattern in a predetermined area of ​​the body, including the following steps: The image of the video frame passes through the first ConvBNRelu6 operator and outputs the first feature map P1; The first feature map passes through the second ConvBNRelu6 operator to output a second feature map P2; The second feature map P2 is sequentially processed by the first C2f operator, the third ConvBNRelu6 operator, and the second C2f operator to output a third feature map P3; The third feature map P3 passes through the fourth ConvBNRelu6 operator, the third C2f operator, the SPPFM operator and the fifth ConvBNRelu6 layer in sequence, and outputs the fourth feature map N2; N2=ConvBNRelu6(SPPFM(C2f(ConvBNRelu6(P3)))) The SPPFM operator includes a MaxPool2d module and an AvgPool2d module. The feature map output by the SPPFM operator is: a feature map output by the MaxPool2d module and a feature map output by the AvgPool2d module are added in the channel dimension, and then subjected to a ConvBNRelu6 operator operation. The fourth feature map N2 is upsampled by the UpSample operator, and is concatenated with the third feature map P3 by the first Concat operator. The output of the first Concat operator is then sequentially concatenated by the CBAM-LSA operator and the fourth C2f operator to output the fifth feature map N1. N1=C2f(CBAM_LSA(Concat(UpSample(N2),P3))) The CBAM-LSA operator includes: a CBAM_LSA channel attention module and a CBAM_LSA spatial attention module, wherein: y1=Clamp(alpha*Conv2d(AvgPool2d(x))+beta) y2=Clamp(alpha*Conv2d(y1)+beta) Where y1 is the output of the CBAM_LSA channel attention module, y2 is the output of the CBAM_LSA spatial attention module, alpha is a constant of 0.1, beta is a constant of 0.5, and Clamp is a numerical truncation operator. When the output value is lower than 0, its value is set to 0, and when the output value is higher than 1, its value is set to 1. The fifth feature map N1 is sequentially processed by the first Conv2d operator and the first Sigmoid operator to output a sixth feature map O1; O1=Sigmoid(Conv2d(N1)) Among them, after the fifth feature map N1 is copied, it passes through the sixth ConvBNRelu6 operator, and then passes through the second Concat operator, the second CBAM-LSA operator, and the fifth C2f operator together with the fourth feature map N2 in sequence to output the seventh feature map N3; N3=C2f(CBAM_LSA(Concat(ConvBNRelu6(N1),N2))) The seventh feature map N3 passes through the second Conv2d operator and the second Sigmoid layer in sequence to output the eighth feature map O2; O2=Sigmoid(Conv2d(N3)) Merging the sixth feature map O1 and the eighth feature map O2, extracting a position of a detection frame corresponding to the predetermined body region from the merged feature map, and obtaining a confidence score for the presence of the predetermined label pattern in the detection frame; When the confidence level is greater than a predetermined confidence level threshold, it is determined that the predetermined label image exists in the frame.

2. The personnel detection method according to claim 1, characterized in that: The calculation process of the SPPFM operator includes: M1=MaxPool2d(x) M2=MaxPool2d(M1) M3=MaxPool2d(M2) MS=Concat(M1,M2,M3) A1=AvgPool2d(x) A2=AvgPool2d(A1) A3=AvgPool2d(A2) AS=Concat(A1,A2,A3) MAS=Add(MS,AS) y = ConvBNRelu6(MAS) Among them, x represents the input feature map; M1, M2, M3, MS, A1, A2, A3, AS and MAS represent the output intermediate feature maps; Add represents the operator that adds two feature maps in the channel dimension; y is the feature map output by the SPPFM operator.

3. The personnel detection method according to claim 1, wherein: The training samples for training the target detection network structure are obtained by the following steps: Pre-set difficult negative samples for target detection, wherein the difficult negative samples include: patterns displayed on a mobile phone screen in the environment, patterns on the chest of a person wearing clothes, and / or patterns in the surrounding environment; Randomly sample from the hard negative samples and positive samples, and concatenate at least one positive sample and at least one hard negative sample to form a training sample set.

4. The personnel detection method according to claim 1, wherein: The difficult negative samples and positive samples are spliced ​​according to a predetermined quantity ratio.

5. The personnel detection method according to claim 4, characterized in that: The predetermined quantity ratio includes: 1 to 2, 2 to 1 and 1 to 1.

6. The method for detecting people according to claim 1, wherein: Also includes one or more of the following: The predetermined label pattern is: a pattern of alternating black and white stripes or alternating red and white stripes; The predetermined body area includes a predetermined shoulder area and / or a predetermined chest area.

7. The person detection method according to claim 1, characterized in that: The personnel with the predetermined attributes are new employees in the room.

8. The method for detecting people according to claim 4, wherein: Whether the target person has a predetermined label pattern in a predetermined area of ​​the body is identified by detecting whether the pattern of the target person's shoulder badge and / or chest badge is the predetermined label pattern.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the steps of the personnel detection method according to any one of claims 1 to 8.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the personnel detection method according to any one of claims 1 to 8 are implemented.

Images