Face detection method, device, electronic device and readable storage medium

Abstract

The disclosed embodiments disclose a face detection method, device, electronic device and readable storage medium. The face detection method includes: processing face image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes multiple processing stages, each of which outputs a first feature map; processing multiple first feature maps through a feature fusion network to obtain multiple second feature maps; determining multiple prediction boxes based on the multiple second feature maps; obtaining a first threshold indicating confidence under a flexible maximum operation; converting the first threshold into a second threshold indicating confidence under an addition and subtraction operation; and determining a prediction result based on the prediction box and the second threshold. By converting the threshold comparison of the flexible maximum value into a threshold comparison of an addition and subtraction operation, the amount of calculation is greatly saved, the processing efficiency is improved, and the difficulty of deploying the model on the terminal is reduced.

Description

[0001] Description of the case

[0002] The present disclosure is a divisional application of the invention patent application with application number 202110202066.9 filed on February 23, 2021, and invention name “Face detection method, device, electronic device and readable storage medium”. Technical Field

[0003] The present disclosure relates to the field of face recognition technology, and in particular to a face detection method, device, electronic device and readable storage medium. Background Art

[0004] With the continuous development of human identity verification technology and image intelligent detection and recognition technology, face recognition technology is becoming more and more mature, and face recognition applications are increasingly moving towards terminals and web terminals, and the reduction of input conditions makes human-computer interaction more convenient. Generally speaking, face recognition technology includes face detection technology, face key point positioning technology, face feature extraction technology, and face attribute analysis technology. The inventors have found that the existing face detection algorithm has a large amount of calculation, is time-consuming, and is difficult to deploy in terminals (such as access control). Summary of the invention

[0005] In order to solve the problems in the related art, the embodiments of the present disclosure provide a face detection method, device, electronic device and readable storage medium.

[0006] In a first aspect, a face detection method is provided in an embodiment of the present disclosure.

[0007] Specifically, the face detection method comprises:

[0008] Processing the facial image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, each of the processing stages outputting a first feature map;

[0009] Processing the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps;

[0010] Based on the multiple second feature maps, determining multiple prediction boxes;

[0011] Obtaining a first threshold indicating confidence under a flexible maximum operation;

[0012] converting the first threshold into a second threshold indicating a confidence level under addition and subtraction operations;

[0013] A prediction result is determined based on the prediction box and the second threshold.

[0014] In combination with the first aspect, in a first implementation of the first aspect of the present disclosure, the first threshold is converted into the second threshold by the following formula:

[0015] Wherein, t1 is the first threshold, and t2 is the second threshold.

[0016] In combination with the first aspect, in a second implementation of the first aspect of the present disclosure, the backbone convolutional neural network includes a plurality of ordinary convolutional layers and a plurality of depth-separable convolutional layers that are alternately arranged.

[0017] In combination with the first aspect, any one of the first or second implementation manners of the first aspect, in a third implementation manner of the first aspect of the present disclosure, processing the plurality of first feature maps through a feature fusion network to obtain the plurality of second feature maps includes:

[0018] Processing the plurality of first feature maps through a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps;

[0019] The multiple third feature maps are processed respectively by the second fusion sub-network to obtain multiple second feature maps.

[0020] In combination with the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect of the present disclosure, the plurality of first feature maps include at least a feature map C1 and a feature map C2, the size of the feature map C1 is larger than the size of the feature map C2, and the plurality of first feature maps are processed by a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps including:

[0021] The feature maps C1 and C2 are processed by 1×1 convolution layers respectively to obtain feature maps M1 and P2 with the same number of channels;

[0022] Upsample P2 to obtain feature map M2_up, and superimpose it with M1 to obtain feature map M1_add;

[0023] Process M1_add through a 3×3 convolutional layer to obtain the feature map P1.

[0024] Among them, P1 and P2 are the third feature maps, and the upsampling is implemented using the nearest neighbor interpolation algorithm.

[0025] In combination with the third implementation manner of the first aspect, in a fifth implementation manner of the first aspect of the present disclosure, the processing of the multiple third feature maps respectively by the second fusion sub-network to obtain the multiple second feature maps includes performing the following operations on each third feature map P:

[0026] Process P through a convolutional layer with the first number of output channels to obtain a feature map S1;

[0027] Process P through a convolutional layer with a second number of output channels to obtain a feature map T;

[0028] Process T through two convolution channels respectively to obtain feature maps S2 and S3, where the number of channels of S2 and S3 is the number of the second output channels;

[0029] S1, S2 and S3 are superimposed channel by channel to obtain a second feature map F with a predetermined number of channels.

[0030] In combination with the first aspect and any one of the first to fifth implementations of the first aspect, in a sixth implementation of the first aspect of the present disclosure, determining multiple prediction frames based on the multiple second feature maps includes:

[0031] Determine an anchor point position based on the pixel points in the second feature map;

[0032] Determining a size of the anchor box based on a size of the second feature map, wherein the size of the second feature map is negatively correlated with the size of the anchor box;

[0033] Determine the density of the prediction box generated at each anchor point position based on the downsampling ratio of the second feature map and the size of the anchor point box;

[0034] Based on the anchor point position, the size of the anchor point box, and the density of the prediction boxes generated by each anchor point position, multiple prediction boxes are determined, each of the prediction boxes including the following information: the position and size of the prediction box, a first confidence that the prediction box is a positive sample, and a second confidence that the prediction box is a negative sample.

[0035] In combination with the sixth implementation manner of the first aspect, in a seventh implementation manner of the first aspect of the present disclosure, determining the prediction result based on the prediction frame and the second threshold includes:

[0036] Based on the difference between the first confidence level and the second confidence level, determining a prediction box having a prediction score greater than a second threshold;

[0037] The prediction boxes whose prediction scores are greater than the second threshold are sorted according to the prediction scores by using a binary tree interpolation and in-order sorting algorithm to obtain a sorting result;

[0038] According to the sorting result, the prediction boxes whose prediction scores are greater than the second threshold are processed by non-maximum suppression to filter out repeated prediction boxes and obtain a prediction result.

[0039] In combination with the first aspect and any one of the first to seventh implementations of the first aspect, in an eighth implementation of the first aspect of the present disclosure, the method further includes:

[0040] Obtain a sample image;

[0041] Mapping the brightness of the sample image to a specific interval to construct an augmented image;

[0042] A face detection model including the backbone convolutional neural network and the feature fusion network is trained based on the sample image and the augmented image.

[0043] In a second aspect, a face detection device is provided in an embodiment of the present disclosure.

[0044] Specifically, the face detection device comprises:

[0045] A feature extraction module is configured to process the facial image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, each of the processing stages outputting a first feature map;

[0046] A feature fusion module is configured to process the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps;

[0047] A prediction box determination module is configured to determine a plurality of prediction boxes based on the plurality of second feature maps;

[0048] A threshold acquisition module is configured to acquire a first threshold indicating a confidence level under a flexible maximum operation;

[0049] A threshold conversion module, configured to convert the first threshold into a second threshold indicating a confidence level under addition and subtraction operations;

[0050] A result determination module is configured to determine a prediction result based on the prediction box and the second threshold.

[0051] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspect and the first to eighth implementations of the first aspect.

[0052] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, on which computer instructions are stored. When the computer instructions are executed by a processor, a method as described in any one of the first aspect and the first to eighth implementations of the first aspect is implemented.

[0053] According to the technical solution provided by the embodiment of the present disclosure, by converting the threshold comparison of the flexible maximum value into the threshold comparison of addition and subtraction operations, the amount of calculation is greatly saved, the processing efficiency is improved, and the difficulty of deploying the model on the terminal is reduced.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0056] Figure 1 A flow chart showing a face detection method according to an embodiment of the present disclosure is shown;

[0057] Figure 2 A flowchart showing a plurality of first feature maps is obtained by processing the plurality of first feature maps through a feature fusion network according to an embodiment of the present disclosure;

[0058] Figure 3 A flowchart showing a plurality of first feature maps is obtained by processing the plurality of first feature maps through a first fusion sub-network according to an embodiment of the present disclosure;

[0059] Figure 4 A flowchart showing a plurality of second feature maps is obtained by respectively processing the plurality of third feature maps through a second fusion sub-network according to an embodiment of the present disclosure;

[0060] Figure 5 A schematic diagram showing the structure of a second fusion sub-network according to an embodiment of the present disclosure is shown;

[0061] Figure 6 A flowchart of determining a plurality of prediction boxes based on the plurality of second feature maps according to an embodiment of the present disclosure is shown;

[0062] Figure 7A to Figure 7D A schematic diagram showing anchor point box expansion according to an embodiment of the present disclosure;

[0063] Figure 8 A flowchart showing a prediction result determined based on the prediction box and the second threshold according to an embodiment of the present disclosure is shown;

[0064] Fig. 9 A flow chart showing a model training method according to an embodiment of the present disclosure;

[0065] Fig.10 A block diagram of a face detection device according to an embodiment of the present disclosure is shown;

[0066] Fig.11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0067] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing a face detection method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0068] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0069] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0070] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0071] Figure 1 A flow chart of a face detection method according to an embodiment of the present disclosure is shown.

[0072] like Figure 1 As shown, the method includes operations S110 to S160.

[0073] In operation S110, the face image data is processed by a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, and each of the processing stages outputs a first feature map;

[0074] In operation S120, a plurality of the first feature maps are processed by a feature fusion network to obtain a plurality of second feature maps;

[0075] In operation S130, a plurality of prediction boxes are determined based on the plurality of second feature maps;

[0076] In operation S140, a first threshold indicating a confidence level under a flexible maximum operation is obtained;

[0077] In operation S150, the first threshold is converted into a second threshold indicating a confidence level under addition and subtraction operations;

[0078] In operation S160, a prediction result is determined based on the prediction box and the second threshold.

[0079] According to the technical solution of the embodiment of the present disclosure, by converting the threshold comparison of the flexible maximum value into the threshold comparison of addition and subtraction operations, the amount of calculation is greatly saved, the processing efficiency is improved, and the difficulty of deploying the model on the terminal is reduced.

[0080] According to the embodiments of the present disclosure, the face image data may be the image data of a single image or the image data of a video frame. The face image data may be collected by a camera, for example, the access control system may collect the image data collected by the camera, and use the face recognition algorithm to identify whether the image contains the face of an authorized person, thereby controlling the state of the access control.

[0081] According to the disclosed embodiment, the backbone convolutional neural network can, for example, adopt various existing neural network models to extract image features. The backbone convolutional neural network is divided into multiple processing stages, and each processing stage outputs a first feature map after execution. Optionally, the backbone convolutional neural network gradually reduces the size of data in the direction from the input layer to the output layer, so the size of the first feature map output in the previous processing stage is larger than the size of the first feature map output in the subsequent processing stage, thereby forming a feature pyramid.

[0082] According to an embodiment of the present disclosure, a feature fusion network is used to further mine features in multiple first feature maps to form multiple second feature maps. The obtained second feature maps are used to generate multiple prediction boxes, each of which may include the following information: the position and size of the prediction box, the first confidence that the prediction box is a positive sample, and the second confidence that the prediction box is a negative sample.

[0083] Usually, a large number of prediction boxes are generated in this step. In order to obtain the final prediction results, it is necessary to screen out a small number of prediction boxes that meet certain conditions from the numerous prediction boxes.

[0084] According to common understanding, the sum of the first confidence and the second confidence should be 1, but since the first confidence and the second confidence are generated independently and there is no constraint relationship between them, it is possible that the sum of the first confidence and the second confidence is greater than 1 or less than 1. For example, if the first confidence is 0.2 and the second confidence is 0.1, in this case, it is wrong to directly use the first confidence to conclude that the probability of the predicted box being a positive sample is very small.

[0085] In dealing with similar problems, some related technologies perform a softmax operation on two confidences. For example, when the first confidence is 0.2 and the second confidence is 0.1, the corrected first confidence is calculated as The corrected second confidence level is After all the objects to be identified are processed as above, each object can be uniformly measured by a threshold. The inventors found that this method has a large amount of calculation and its performance is not satisfactory when facing the problem of screening a large number of prediction boxes.

[0086] The disclosed embodiment proposes a method of measuring whether the prediction box meets the preset conditions through the threshold of addition and subtraction operations. For example, assuming that the first threshold set in the above softmax method is 0.6, in order to select the data whose first confidence is greater than 0.6 after the softmax processing, the method of the disclosed embodiment converts it into a comparison method in which the difference between the first confidence and the second confidence is greater than 0.4. For example, for the case where the first confidence is 0.2 and the second confidence is 0.1, the difference between the two is 0.1, which is not greater than 0.4 and does not meet the conditions. For the case where the first confidence is 0.6 and the second confidence is 0.1, the difference between the two is 0.5, which is greater than 0.4 and meets the conditions. In this way, the exponential operation is converted into addition and subtraction operations, which greatly reduces the amount of calculation and can be used to implement an algorithm for fast face detection in a terminal.

[0087] According to the embodiment of the present disclosure, the first threshold value may be converted into the second threshold value by the following formula:

[0088]

[0089] Wherein, t1 is the first threshold, and t2 is the second threshold.

[0090] The rationality of this method is explained below.

[0091] According to the softmax formula:

[0092]

[0093] Among them, a1 is the first confidence level, a0 is the second confidence level, S1 is the corrected first confidence level, and S0 is the corrected second confidence level.

[0094] If S1 is greater than the first threshold t1, then due to You can get:

[0095]

[0096] The above formula is equivalent to Since the ln function is monotonically increasing, the inequality still holds when taking logarithms on both sides, that is:

[0097] ln(1-t1)+a1>ln(t1)+a0

[0098] When the first threshold t1 is determined, ln(t1)-ln(1-t1) is a constant, and the constant is set to t2, that is:

[0099]

[0100] When the first threshold t1 is fixed, the value t2 only needs to be calculated once, and the difference between the first confidence a1 and the second confidence a0 can be used to directly calculate whether the score meets the condition.

[0101] The method provided by the embodiment of the present disclosure cleverly compares the softmax threshold by calculating a constant once, and the effect of the two is consistent compared with the method of calculating softmax for each object. When the size of the input network is fixed, for example, 8208 prediction boxes can be generated. If softmax is calculated for each prediction box, 8208 softmax operations need to be calculated. However, this solution only needs to calculate the second threshold once, which can simplify a large number of softmax operations into addition and subtraction operations. Therefore, the method of the embodiment of the present disclosure can greatly reduce the amount of calculation in the data post-processing stage and improve the detection efficiency.

[0102] According to an embodiment of the present disclosure, the backbone convolutional neural network includes a plurality of ordinary convolutional layers and a plurality of depth-separable convolutional layers that are alternately arranged.

[0103] By using depthwise separable convolution instead of ordinary convolution to extract features, the number of parameters can be further reduced while ensuring sufficient network depth, thereby reducing the amount of computation.

[0104] Assume that the convolution kernel size is K h ×K w , the number of input channels is C in , the number of output channels is C out , the width and height of the output feature map are W and H respectively, and the bias term is ignored here.

[0105] For a standard convolutional layer:

[0106] The number of parameters is: params = K h ×K w ×C in ×C out

[0107] The number of floating point operations performed per second FLOPs is: params×H×W

[0108] And for depthwise separable convolution:

[0109] When performing depthwise convolution, only one convolution kernel with dimension in_channels is used for feature extraction (no feature combination is performed); when performing pointwise convolution, only one convolution kernel with dimension in_channels 1×1 of output_channels is used for feature combination, and they are added according to different proportions (learnable parameters).

[0110] The number of parameters is increased from the original Kh ×K w ×C in ×C out Became K h ×K w ×C in ×1+1×1×C in ×C out , if K h =3, K w =3, C out =64, the number of parameters will be reduced to about 1 / 8 to 1 / 9 of the original.

[0111] According to the embodiment of the present disclosure, the backbone neural network structure can be implemented as shown in Table 1, for example. Conv represents a normal convolution layer followed by a batch normalization layer (Batch Normalization, BN) and an activation layer (such as ReLU), Convdw represents a depth-separable convolution layer followed by a batch normalization layer and an activation layer, S2 represents a step size of 2, S1 represents a step size of 1, and the padding mode is the SAME mode. C1, C2, and C3 are the outputs of the three branches, that is, the three first feature maps.

[0112] Table 1

[0113] Convolution type / stride Convolution kernel size Input size Conv / S2 3×3×3×8 192×128×3 Conv dw / S1 3×3×8dw 96×64×8 Conv / S1 1×1×8×16 96×64×8 Conv dw / S2 3×3×16dw 96×64×16 Conv / S1 1×1×16×32 48×32×16 Conv dw / S1 3×3×32dw 48×32×32 Conv / S1 1×1×32×32 48×32×32 Conv dw / S2 3×3×32dw 48×32×32 Conv / S1 1×1×32×64 24×16×32 Conv dw / S1 3×3×64dw 24×16×64 Conv / S1(C1) 1×1×64×64 24×16×64 Conv dw / S2 3×3×64dw 24×16×64 Conv / S1 1×1×64×128 12×8×64 Conv dw / S1 3×3×128dw 12×8×128 Conv / S1 1×1×128×128 12×8×128 Conv dw / S1 3×3×128dw 12×8×128 Conv / S1 1×1×128×128 12×8×128 Conv dw / S1 3×3×128dw 12×8×128 Conv / S1 1×1×128×128 12×8×128 Conv dw / S1 3×3×128dw 12×8×128 Conv / S1(C2) 1×1×128×128 12×8×128 Conv dw / S2 3×3×128dw 12×8×128 Conv / S1 1×1×128×256 6×4×128 Conv dw / S1 3×3×256dw 6×4×256 Conv / S1(C3) 1×1×256×256 6×4×256

[0114] According to an embodiment of the present disclosure, the feature fusion network may include, for example, a first fusion sub-network and a second fusion sub-network.

[0115] Figure 2 A flowchart is shown of processing a plurality of the first feature maps through a feature fusion network to obtain a plurality of second feature maps according to an embodiment of the present disclosure.

[0116] like Figure 2 As shown, operation S120 may include operations S210 and S220.

[0117] In operation S210, a plurality of the first feature maps are processed by a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps;

[0118] In operation S220, the plurality of third feature maps are respectively processed by a second fusion sub-network to obtain a plurality of second feature maps.

[0119] Through the two fusions, different receptive fields can be added, faces of different sizes can be noticed, which helps to improve the face detection effect.

[0120] According to an embodiment of the present disclosure, the plurality of first feature maps at least include a feature map C1 and a feature map C2, and the size of the feature map C1 is larger than the size of the feature map C2. As described above, the first feature map may include C1, C2 and C3, for example.

[0121] Figure 3 A flowchart is shown of processing a plurality of the first feature maps through a first fusion sub-network to obtain a plurality of third feature maps according to an embodiment of the present disclosure.

[0122] like Figure 3 As shown, operation S210 may include operations S310 to S330.

[0123] In operation S310, the feature maps C1 and C2 are processed by a 1×1 convolution layer to obtain feature maps M1 and P2 with the same number of channels;

[0124] In operation S320, P2 is upsampled to obtain a feature map M2_up, and superimposed with M1 to obtain a feature map M1_add;

[0125] In operation S330, M1_add is processed through a 3×3 convolution layer to obtain a feature map P1, where P1 and P2 are the third feature map.

[0126] According to the embodiment of the present disclosure, the upsampling can be achieved, for example, by a nearest neighbor interpolation algorithm. As shown below: the left side is a small feature map, and the right side is a feature map obtained by the nearest neighbor interpolation with upsampling twice:

[0127]

[0128]

[0129] Through the nearest neighbor interpolation algorithm, the semantic information of the feature map can be retained to the greatest extent during the upsampling process (which is beneficial for classification), and then fused with the corresponding feature map with rich spatial information (high resolution, which is beneficial for positioning) during the upsampling process, thereby obtaining a feature map with both good spatial information and strong semantic information.

[0130] In one implementation of the present disclosure, the first fusion sub-network may be in a form shown in Table 2, for example.

[0131] Table 2

[0132]

[0133] According to the embodiment of the present disclosure, after the backbone network obtains the outputs of the three branches C1, C2 and C3, they are sent to the feature fusion network to fuse the top-level features with the bottom-level features. Feature fusion is a top-down process that enlarges the small feature map of the top layer to the feature map of the previous stage by up-sampling. Figure 1 Sample size.

[0134] As shown in Table 2, the C1, C2, and C3 layers are first subjected to 1×1 convolution to change the number of channels of the feature map (the number of channels of all M layers is consistent, for example, d=64 is set in the embodiment of the present disclosure) to obtain M1, M2, and M3. M3 is obtained by upsampling to obtain M3_up, and then added to the corresponding position of M2 to obtain M2_add. M2_add is obtained by upsampling to obtain M2_up, and then added to the corresponding position of M1 to obtain M1_add. The feature maps of the M1_add and M2_add layers are then subjected to 3×3 convolution (to reduce the aliasing effect caused by the nearest neighbor interpolation, and the surrounding numbers are the same) to obtain the final P1, P2, and P3 layer features, that is, the third feature map.

[0135] It can be understood that C2, C3, M2, P3 (M3), M3_up, M2_add and P2 in the embodiment shown in Table 2 are respectively equivalent to C1, C2, M1, P2, M2_up, M1_add and P1 described in operations S310 to S330. The reason is that there are three first feature maps in the embodiment shown in Table 2, which leads to the shift of the sequence number.

[0136] According to the embodiment of the present disclosure, after the feature fusion layer, P1, P2, and P3 are further fused with information of different receptive fields through a second fusion sub-network.

[0137] Figure 4 A flowchart is shown of respectively processing the multiple third feature maps through a second fusion sub-network to obtain multiple second feature maps according to an embodiment of the present disclosure.

[0138] like Figure 4 As shown, operation S220 may include operations S410 to S440.

[0139] In operation S410, P is processed by a convolutional layer having a first number of output channels to obtain a feature map S1;

[0140] In operation S420, P is processed by a convolutional layer having a second number of output channels to obtain a feature map T;

[0141] In operation S430, T is processed through two convolution channels respectively to obtain feature maps S2 and S3, wherein the number of channels of S2 and S3 is the second output channel number;

[0142] In operation S440, S1, S2, and S3 are superimposed channel by channel to obtain a second feature map F having a predetermined number of channels.

[0143] Refer to the following Figure 5 The second fusion sub-network shown is explained.

[0144] Figure 5 A schematic diagram of the structure of a second fusion sub-network according to an embodiment of the present disclosure is shown.

[0145] like Figure 5 As shown, the second fusion subnetwork can be a three-branch network. For any third feature map, such as P1, first perform a normal convolution with a convolution kernel size of 3×3 and an output channel of 32, and then perform batch normalization to obtain S1; P1 then passes through a convolution kernel of 3×3 with an output channel of 16, a batch normalization layer, and a Relu activation layer to obtain T2, and T2 then passes through a convolution kernel of 3×3 with an output channel of 16, a batch normalization layer, and a Relu activation layer to obtain S2; T2 passes through another convolution kernel of 3×3 with an output channel of 16, a batch normalization layer, and a Relu activation layer to obtain T3, and T3 passes through a convolution kernel of 3×3 with an output channel of 16, a batch normalization layer, and a Relu activation layer to obtain S3. Finally, S1, S2, and S3 are concatenated according to the channel dimension and then passed through a Relu activation layer to obtain the second feature map F1, which is still a feature map of 64 channels. Similarly, the above processing can be performed on P2 and P3 to obtain F2 and F3 respectively.

[0146] Similarly, P1 and T2 in the above-illustrated embodiment correspond to P and T described in operations S410 to S440, respectively.

[0147] The inventors have found that the existing detection algorithms still have the problem of poor detection effect on smaller faces. To this end, the embodiments of the present disclosure propose a solution for expanding the number of anchor boxes under a smaller field of view when generating a prediction box.

[0148] Figure 6 A flowchart of determining a plurality of prediction boxes based on the plurality of second feature maps according to an embodiment of the present disclosure is shown.

[0149] like Figure 6 As shown, operation S130 may include operations S610 to S640.

[0150] In operation S610, an anchor point position is determined based on the pixel points in the second feature map;

[0151] In operation S620, a size of the anchor box is determined based on a size of the second feature map, wherein the size of the second feature map is negatively correlated with the size of the anchor box;

[0152] In operation S630, the density of the prediction box generated at each anchor point position is determined based on the downsampling magnification of the second feature map and the size of the anchor point box;

[0153] In operation S640, a plurality of prediction boxes are determined based on the anchor point positions, the sizes of the anchor boxes, and the densities of the prediction boxes generated at the respective anchor point positions.

[0154] According to an embodiment of the present disclosure, the downsampling ratios between the sizes of the second feature maps F1, F2, and F3 and the original input image are, for example, 8, 16, and 32, respectively. As the number of convolutional layers passed is increased, the downsampling ratio is higher, and the corresponding receptive field is also larger. The present disclosure embodiment can define anchor boxes of various sizes, for example, anchor boxes of 16×16, 32×32, 64×64, 128×128, 256×256, and 512×512 can be defined. A smaller-sized anchor box is used for prediction on a second feature map with a smaller downsampling ratio (a second feature map with a larger size), and a larger-sized anchor box is used for prediction on a second feature map with a larger downsampling ratio (a second feature map with a smaller size). For example, two sizes of anchor box predictions are used on F1: 16×16 and 32×32, two sizes of anchor box predictions are used on F2: 64×64 and 128×128, and two sizes of anchor box predictions are used on F3: 256×256 and 512×512.

[0155] Typically, each anchor position predicts a set of anchor boxes of different aspect ratios and sizes, for example, when the aspect ratios are 1:1, 1:1.5 and 1.5:1 and the sizes are 16×16 and 32×32, each anchor position predicts 3×2=6 anchor boxes. In the disclosed embodiment, the aspect ratio is fixed to 1:1, and for feature map F1, each anchor position predicts two anchor boxes of 16×16 and 32×32, while for feature map F3, each anchor position predicts two anchor boxes of 256×256 and 512×512.

[0156] Obviously, in this case, the smaller anchor boxes will be sparser than the larger anchor boxes. This is also an important reason why the small target detection effect is not good. In order to solve the above problem, the embodiment of the present disclosure proposes to increase the density of the anchor boxes of some feature maps. Increasing the density of anchor boxes means that at each anchor position, for each combination of aspect ratio and anchor box size, multiple anchor boxes are predicted. For example, the anchor boxes with insufficient density can be offset and doubled by the center, such as Figure 7A-7D shown.

[0157] It can be defined that the density of the anchor box of size 16×16 is increased by 4 times, the density of the anchor box of size 32×32 is increased by 2 times, and the density of the anchor box of size 64×64 is increased by 2 times, so that each pixel position of the F1 feature layer obtains 4×4+2×2=20 prediction offsets (16 16×16 anchor boxes and 4 32×32 anchor boxes), and each pixel position of the F2 feature layer predicts 2×2+1×1=5 prediction offsets (4 64×64 anchor boxes and 1 128×128 anchor box), and each pixel position of the F3 feature layer predicts 1×1+1×1=2 prediction offsets (1 256×256 anchor box and 1 512×512 anchor box).

[0158] Table 3 illustrates the selection of anchor boxes of different sizes, the density corresponding to each anchor box, and the ratio of the step length of the anchor box to the size of the anchor box according to an embodiment of the present disclosure.

[0159] Table 3

[0160] Anchor box size Downsampling ratio Anchor box density Anchor box step Proportion 16×16 8 N＝4 2＝8 / 4 2 / 16＝1 / 8 32×32 8 N＝2 4＝8 / 2 4 / 32＝1 / 8 64×64 16 N＝2 8＝16 / 2 8 / 64＝1 / 8 128×128 16 N＝1 16＝16 / 1 16 / 128＝1 / 8 256×256 32 N＝1 32＝32 / 1 32 / 256＝1 / 8 512×512 32 N＝1 32＝32 / 1 32 / 512＝1 / 16

[0161] As shown in Table 3, the step size of the anchor box is defined as the downsampling ratio divided by the density of the anchor box, and the step size of the anchor box divided by the size ratio of the anchor box is defined as a fixed value (for example, 1 / 8, where the anchor box size is 512×512, in order to ensure that the density of the anchor box is not less than 1, the ratio is determined to be 1 / 16).

[0162] That is, the size of the F1 feature map is 24×16, each pixel is responsible for predicting 4×4 anchor boxes of size 16 and 2×2 anchor boxes of size 32, and each pixel gets 20 prediction boxes; the size of the F2 feature map is 12×8, each pixel is responsible for predicting 2×2 anchor boxes of size 64 and 1×1 anchor box of size 128, and each pixel gets 5 prediction boxes; the size of the F3 feature map is 6×4, each pixel is responsible for predicting 1×1 anchor box of size 256 and 1×1 anchor box of size 512, and each pixel gets 2 prediction boxes. A total of 24×16×20+12×8×5+6×4×2=8208 prediction boxes are generated.

[0163] According to an embodiment of the present disclosure, each of the prediction boxes includes the following information: the position and size of the prediction box, the first confidence that the prediction box is a positive sample, and the second confidence that the prediction box is a negative sample, for example, it can be expressed as (x, y, w, h, a1, a0), wherein x, y are the offsets of the prediction box relative to the anchor point, w, h are the changes of the prediction box relative to the anchor point box, a1, a0 are the first confidence of the positive sample and the second confidence of the prediction box for the negative sample, and these data are generated based on the second feature map.

[0164] The method of the disclosed embodiment increases the density of anchor boxes used to predict smaller faces, thereby improving the detection rate of smaller faces.

[0165] Figure 8 A flowchart of determining a prediction result based on the prediction box and the second threshold according to an embodiment of the present disclosure is shown.

[0166] like Figure 8 As shown, operation S160 may include operations S810 to S830.

[0167] In operation S810, based on the difference between the first confidence and the second confidence, a prediction box having a prediction score greater than a second threshold is determined;

[0168] In operation S820, the prediction boxes whose prediction scores are greater than the second threshold are sorted according to the prediction scores by using a binary tree interpolation and in-order sorting algorithm to obtain a sorting result;

[0169] In operation S830, according to the sorting result, the prediction boxes whose prediction scores are greater than the second threshold are processed by non-maximum suppression to filter out repeated prediction boxes to obtain a prediction result.

[0170] According to the disclosed embodiment, after obtaining the prediction results of the F1, F2, and F3 feature layers, the post-processing stage of the prediction frame is entered. For example, the threshold conversion method described above can be used to consider the prediction frames with prediction scores lower than 0.3 as negative samples and filter them out. The prediction frames with prediction scores higher than 0.3 are sorted in descending order according to the prediction scores, and then NMS (non-maximum suppression) is performed to consider the frames with IOU (overlapping area divided by union area) greater than 0.5 as duplicate prediction frames and further filter them out to obtain the prediction results. Optionally, the prediction frames that are retained and have scores greater than the threshold can be further output as prediction results according to another threshold defined by the system (such as 0.8 or 0.5).

[0171] When using NMS (non-maximum suppression) to filter duplicate prediction boxes, you need to sort them according to the scores to retain the prediction boxes with the highest confidence and suppress the duplicate prediction boxes with lower scores. The time complexity of the conventional sorting algorithm is O(n 2 ), the disclosed embodiment can use binary tree interpolation and binary tree in-order search to make the time complexity of sorting between O(Log2n) and O(n), further improving the processing efficiency.

[0172] In addition, the inventors have found that the existing face detection algorithms have a high rate of missed detection and false alarms in complex scenes such as backlight, dark light, and wearing masks. Therefore, the disclosed embodiments propose to alleviate the above problems by augmenting the training samples and improve the face detection rate in complex scenes.

[0173] Fig. 9 A flowchart of a model training method according to an embodiment of the present disclosure is shown.

[0174] like Fig. 9 As shown, this method Figures 1 to 8 Based on any of the illustrated embodiments, operations S910 to S930 may also be included.

[0175] In operation S910, a sample image is obtained;

[0176] In operation S920, the brightness of the sample image is mapped to a specific interval to construct an augmented image;

[0177] In operation S930, a face detection model including the backbone convolutional neural network and the feature fusion network is trained based on the sample image and the augmented image.

[0178] According to an embodiment of the present disclosure, the RGB data of an input video frame or image may be preprocessed. For example, the brightness value of each pixel in the image may be processed by linear stretching:

[0179] y=(x-min(x))×(dmax-dmin)) / (max(x)-min(x)+1.0)+dmin

[0180] Where x represents the input image, min(x) represents the minimum pixel brightness in the image, max(x) represents the maximum pixel brightness in the image, and dmax and dmin represent the maximum and minimum values ​​of mapping the image to the target area, respectively.

[0181] In order to enhance the data for backlit situations, the brightness of the training images is adjusted using a maximum and minimum linear stretching method, and the brightness of the training data is randomly mapped to a specific range, such as 155-255, so that a certain amount of overexposure or backlit data appears in the training samples, so that the trained model can have better generalization capabilities for backlit and strong light data.

[0182] Similarly, brightness can also be mapped to a smaller numerical range to enhance the model's generalization ability for data in dark scenes.

[0183] The disclosed embodiment provides a face detection method based on a deep separable convolutional neural network. The RGB data of a video frame or image is input through a human-computer interface, and the data is normalized and preprocessed. The preprocessed data is input into a backbone convolutional neural network to obtain preliminary feature maps at three different stages. The feature maps at different stages are then input into a feature fusion layer for feature fusion and three fused feature maps are output. Each feature layer is then subjected to a three-branch convolutional neural network to further extract features, and each feature layer is used to predict prediction frames of different sizes. Finally, negative samples and prediction frames with high overlap are filtered out through post-processing, and prediction frames with confidence scores higher than a threshold are output. This method can reduce the amount of computation and improve the detection capability of smaller faces.

[0184] Fig.10 A block diagram of a face detection device according to an embodiment of the present disclosure is shown. The device 1000 can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0185] like Fig.10 As shown, the information processing device 1000 includes a feature extraction module 1010 , a feature fusion module 1020 , a prediction box determination module 1030 , a threshold acquisition module 1040 , a threshold conversion module 1050 and a result determination module 1060 .

[0186] The feature extraction module 1010 is configured to process the face image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, and each of the processing stages outputs a first feature map;

[0187] A feature fusion module 1020 is configured to process the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps;

[0188] A prediction box determination module 1030 is configured to determine a plurality of prediction boxes based on the plurality of second feature maps;

[0189] The threshold acquisition module 1040 is configured to acquire a first threshold indicating a confidence level under a flexible maximum operation;

[0190] A threshold conversion module 1050, configured to convert the first threshold into a second threshold indicating a confidence level under addition and subtraction operations;

[0191] The result determination module 1060 is configured to determine a prediction result based on the prediction box and the second threshold.

[0192] According to an embodiment of the present disclosure, the first threshold is converted into the second threshold by the following formula:

[0193] Wherein, t1 is the first threshold, and t2 is the second threshold.

[0194] According to an embodiment of the present disclosure, the backbone convolutional neural network includes a plurality of ordinary convolutional layers and a plurality of depth-separable convolutional layers that are alternately arranged.

[0195] According to an embodiment of the present disclosure, processing the plurality of first feature maps through a feature fusion network to obtain the plurality of second feature maps includes:

[0196] Processing the plurality of first feature maps through a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps;

[0197] The multiple third feature maps are processed respectively by the second fusion sub-network to obtain multiple second feature maps.

[0198] According to an embodiment of the present disclosure, the plurality of first feature maps include at least a feature map C1 and a feature map C2, the size of the feature map C1 is larger than the size of the feature map C2, and the plurality of first feature maps are processed by the first fusion sub-network to obtain the plurality of third feature maps, including:

[0199] The feature maps C1 and C2 are processed by 1×1 convolution layers respectively to obtain feature maps M1 and P2 with the same number of channels;

[0200] Upsample P2 to obtain feature map M2_up, and superimpose it with M1 to obtain feature map M1_add;

[0201] Process M1_add through a 3×3 convolutional layer to obtain the feature map P1.

[0202] Among them, P1 and P2 are the third feature maps, and the upsampling is implemented using the nearest neighbor interpolation algorithm.

[0203] According to an embodiment of the present disclosure, the processing of the plurality of third feature maps respectively through the second fusion sub-network to obtain the plurality of second feature maps includes performing the following operations on each third feature map P:

[0204] Process P through a convolutional layer with the first number of output channels to obtain a feature map S1;

[0205] Process P through a convolutional layer with a second number of output channels to obtain a feature map T;

[0206] Process T through two convolution channels respectively to obtain feature maps S2 and S3, where the number of channels of S2 and S3 is the number of the second output channels;

[0207] S1, S2 and S3 are superimposed channel by channel to obtain a second feature map F with a predetermined number of channels.

[0208] According to an embodiment of the present disclosure, determining a plurality of prediction frames based on the plurality of second feature maps includes:

[0209] Determine an anchor point position based on the pixel points in the second feature map;

[0210] Determining a size of the anchor box based on a size of the second feature map, wherein the size of the second feature map is negatively correlated with the size of the anchor box;

[0211] Determine the density of the prediction box generated at each anchor point position based on the downsampling ratio of the second feature map and the size of the anchor point box;

[0212] Based on the anchor point position, the size of the anchor point box, and the density of the prediction boxes generated by each anchor point position, multiple prediction boxes are determined, each of the prediction boxes including the following information: the position and size of the prediction box, a first confidence that the prediction box is a positive sample, and a second confidence that the prediction box is a negative sample.

[0213] According to an embodiment of the present disclosure, determining a prediction result based on the prediction frame and the second threshold comprises:

[0214] Based on the difference between the first confidence level and the second confidence level, determining a prediction box having a prediction score greater than a second threshold;

[0215] The prediction boxes whose prediction scores are greater than the second threshold are sorted according to the prediction scores by using a binary tree interpolation and in-order sorting algorithm to obtain a sorting result;

[0216] According to the sorting result, the prediction boxes whose prediction scores are greater than the second threshold are processed by non-maximum suppression to filter out repeated prediction boxes and obtain a prediction result.

[0217] According to an embodiment of the present disclosure, the device may also include a training module configured to obtain a sample image, map the brightness of the sample image to a specific interval to construct an augmented image, and train a face detection model including the backbone convolutional neural network and the feature fusion network based on the sample image and the augmented image.

[0218] The present disclosure also discloses an electronic device, Fig.11 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0219] like Fig.11 As shown, the electronic device 1100 includes a memory 1101 and a processor 1102, wherein the memory 1101 is used to store a program that supports the electronic device to execute the information processing method or code generation method in any of the above embodiments, and the processor 1102 is configured to execute the program stored in the memory 1101.

[0220] According to an embodiment of the present disclosure, the memory 1101 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1102 to implement the face detection method described in any of the above embodiments.

[0221] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing a face detection method according to an embodiment of the present disclosure is shown.

[0222] like Fig.12 As shown, the computer system 1200 includes a processing unit 1201, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the system 1200 are also stored. The processing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0223] The following components are connected to the I / O interface 1205: an input part 1206 including a keyboard, a mouse, etc.; an output part 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1208 including a hard disk, etc.; and a communication part 1209 including a network interface card such as a LAN card, a modem, etc. The communication part 1209 performs communication processing via a network such as the Internet. The drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that the computer program read therefrom is installed into the storage part 1208 as needed. Among them, the processing unit 1201 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0224] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a readable medium thereof, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1209, and / or installed from the removable medium 1211.

[0225] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0226] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0227] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0228] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A face detection method, comprising: Processing the facial image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, each of the processing stages outputting a first feature map; Processing the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps; Determine an anchor point position based on the pixel points in the second feature map; Determining a size of the anchor box based on a size of the second feature map, wherein the size of the second feature map is negatively correlated with the size of the anchor box; Determine the density of the prediction box generated at each anchor point position based on the downsampling ratio of the second feature map and the size of the anchor point box; Determine a plurality of prediction boxes based on the anchor point positions, the sizes of the anchor point boxes, and the densities of the prediction boxes generated by the respective anchor point positions, each of the prediction boxes comprising the following information: the position and size of the prediction box, a first confidence that the prediction box is a positive sample, and a second confidence that the prediction box is a negative sample; Get the threshold indicating the confidence under the flexible maximum operation; Obtaining a first threshold indicating confidence under a flexible maximum operation includes: The first threshold is converted into a second threshold indicating a confidence level under addition and subtraction operations, wherein the first threshold is converted into the second threshold by the following formula: Wherein, t1 is the first threshold, and t2 is the second threshold; Determining a prediction result based on the prediction box and the threshold includes: determining a prediction result based on the prediction box and the second threshold; Determining a prediction result based on the prediction box and the threshold includes: Determining a prediction box having a prediction score greater than a threshold based on a difference between the first confidence level and the second confidence level; The prediction boxes whose prediction scores are greater than the threshold are sorted according to the prediction scores by using a binary tree interpolation and in-order sorting algorithm to obtain a sorting result; According to the sorting result, the prediction boxes whose prediction scores are greater than the threshold are processed by non-maximum suppression to filter out repeated prediction boxes and obtain a prediction result.

2. The method according to claim 1, wherein: The determining of a plurality of prediction frames based on the anchor point positions, the sizes of the anchor point frames, and the density of the prediction frames generated by the anchor point positions includes: According to the size of the anchor box and the density of the prediction box generated at each anchor position, a set of anchor boxes with different aspect ratios and sizes are predicted at each anchor position; A plurality of prediction boxes are determined based on the anchor boxes with different aspect ratios and different sizes.

3. The method according to claim 1, wherein: The determining of a plurality of prediction frames based on the anchor point positions, the sizes of the anchor point frames, and the density of the prediction frames generated by the anchor point positions includes: According to the density of the prediction box generated by each anchor point position and the size of the anchor point box, the offset is doubled with the anchor point position as the center to determine multiple prediction boxes.

4. The method according to claim 1, wherein: The step of determining the density of the prediction frame generated at each anchor point position based on the downsampling ratio of the second feature map and the size of the anchor point frame comprises: Determine the anchor box step size based on a preset constant and the size of the anchor box; Determine a ratio of a downsampling ratio of the second feature map to a step length of the anchor box as a density of the prediction box.

5. The method according to any one of claims 1 to 4, wherein: Processing the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps comprises: Processing the plurality of first feature maps through a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps; Processing the plurality of third feature maps respectively through a second fusion sub-network to obtain a plurality of second feature maps; The plurality of first feature maps at least include a feature map C1 and a feature map C2, the size of the feature map C1 is larger than the size of the feature map C2, and the plurality of first feature maps are processed by a first fusion subnetwork to fuse features between the first feature maps to obtain a plurality of third feature maps including: The feature maps C1 and C2 are processed by a 1×1 convolution layer respectively to obtain feature maps M1 and P2 with the same number of channels; P2 is upsampled to obtain a feature map M2_up, and superimposed with M1 to obtain a feature map M1_add; Process M1_add through a 3×3 convolutional layer to obtain the feature map P1. Wherein, P1 and P2 are third feature maps, and the upsampling is implemented using a nearest neighbor interpolation algorithm; The step of processing the plurality of third feature maps respectively by the second fusion sub-network to obtain the plurality of second feature maps comprises performing the following operations on each third feature map P: Process P through a convolutional layer with a first number of output channels to obtain a feature map S1; process P through a convolutional layer with a second number of output channels to obtain a feature map T; Process T through two convolution channels respectively to obtain feature maps S2 and S3, where the number of channels of S2 and S3 is the number of the second output channels; S1, S2 and S3 are superimposed channel by channel to obtain a second feature map F with a predetermined number of channels.

6. The face detection device according to any one of claims 1 to 5, characterized in that: include: A feature extraction module is configured to process the facial image data through a backbone convolutional neural network, wherein the backbone convolutional neural network includes a plurality of processing stages, each of the processing stages outputting a first feature map; A feature fusion module is configured to process the plurality of first feature maps through a feature fusion network to obtain a plurality of second feature maps; The prediction box determination module is configured to perform the following operations: Determine an anchor point position based on the pixel points in the second feature map; Determining a size of the anchor box based on a size of the second feature map, wherein the size of the second feature map is negatively correlated with the size of the anchor box; Determine the density of the prediction box generated at each anchor point position based on the downsampling ratio of the second feature map and the size of the anchor point box; and determining a plurality of prediction boxes based on the anchor point positions, the sizes of the anchor point boxes, and the density of the prediction boxes generated by the respective anchor point positions, each of the prediction boxes comprising the following information: the position and size of the prediction box, a first confidence that the prediction box is a positive sample, and a second confidence that the prediction box is a negative sample; A threshold acquisition module is configured to acquire a threshold indicating confidence under a flexible maximum operation; and a result determination module is configured to determine a prediction result based on the prediction box and the threshold.

7. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

8. A readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images