Pedestrian rotated bounding box detection method, system, medium and device

Abstract

The application provides a pedestrian rotation frame detection method, system, medium and device, comprising: acquiring a pedestrian detection frame; constructing a small pedestrian detection neural network; setting a positive sample screening strategy of the pedestrian detection neural network; setting a loss function of the pedestrian detection neural network; training the small pedestrian detection neural network based on the pedestrian detection frame, the positive sample screening strategy and the loss function; pruning the trained small pedestrian detection neural network; loading the pruned convolution weight to restore the precision of the light network; and testing the pruning network effect. The application uses a common convolution structure to design a light deep learning network, uses a reparameterization training method and proposes an efficient pruning method suitable for a BN layer, maximally reduces the model calculation amount, and makes the final model deployment simple and easy, while maintaining the original precision under the condition of reducing the original model calculation amount by 40%.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology for artificial intelligence applications, and relates to a pedestrian detection method, particularly a pedestrian rotating frame detection method, system, medium, and device. Background Technology

[0002] In retail customer flow detection applications, terminal devices typically have limited computing power and perform multiple tasks, including detection and attribute analysis. Reducing the computational load and parameter count of the detection model without sacrificing coverage becomes a major challenge. Current lightweight model detection methods often employ end-to-end detection models like the Yolo series to detect horizontal bounding boxes. However, this approach sacrifices detection of distant or tilted targets. Missing or misaligned detection of these targets negatively impacts detection performance and reduces effective coverage. Current rotated bounding box detection algorithms, mostly used in remote sensing, employ a two-stage detection approach. Even in single-stage models, most describe box rotation by predicting angles, typically outputting the box's center point, width, height, and angle. This representation leads to significant quantization errors when the predicted values ​​differ by orders of magnitude due to differences in length and angle, especially when the output is quantized with low precision at the terminal using the same feature map. In addition, the methods commonly used to solve the periodicity of angles and the EoE problem of switching between long and short sides require a large number of channels to represent the angle, which will also bring significant performance loss to the terminal device's calculation.

[0003] Therefore, in existing pedestrian detection technologies, lightweight pedestrian rotating frame detection technology is needed to achieve real-time, accurate, and effective detection with low load in terminal devices. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a pedestrian rotation frame detection method, system, medium and device to solve the problems of reduced coverage of the horizontal frame due to its small model, quantization error caused by the angle prediction method of the rotation frame and large performance loss in the process of human body detection technology.

[0005] To achieve the above and other related objectives, in a first aspect, this application provides a pedestrian rotation box detection method, comprising the following steps: acquiring pedestrian detection boxes; constructing a pedestrian detection neural network; setting a positive sample selection strategy for the pedestrian detection neural network; setting a loss function for the pedestrian detection neural network; training the pedestrian detection neural network based on the pedestrian detection boxes using the positive sample selection strategy and the loss function; pruning the trained pedestrian detection neural network; and retraining the pruned pedestrian detection neural network based on the pedestrian detection boxes using the positive sample selection strategy and the loss function, so as to perform pedestrian detection based on the retrained pedestrian detection neural network.

[0006] In one implementation of the first aspect, obtaining pedestrian detection boxes includes the following steps: obtaining pedestrian detection boxes based on an open-source dataset.

[0007] In one implementation of the first aspect, constructing a pedestrian detection neural network includes the following steps: constructing a tiny-yolov3 network and adding an attention module to the tiny-yolov3 network for feature fusion; setting up a network post-processing module; and constructing the pedestrian detection neural network based on the tiny-yolov3 network and the network post-processing module.

[0008] In one implementation of the first aspect, the positive sample selection strategy of the pedestrian detection neural network includes the following steps: selecting the predicted bounding boxes of the center point of the real bounding box and its eight surrounding points as the initial screening pre-selection box group of the real bounding box; when a pre-selection box is matched with multiple real bounding boxes in multiple initial screening pre-selection box groups, the pre-selection box is only retained in the initial screening pre-selection box group with the largest intersection-union ratio (IU) with the real bounding box, and is deleted from other initial screening pre-selection box groups; calculating the IU between the real bounding box and the matching initial screening pre-selection box group, calculating the IU threshold based on the IU, and selecting candidate positive samples with an IU greater than the IU threshold as the final positive samples.

[0009] In one implementation of the first aspect, setting the loss function of the pedestrian detection neural network includes the following steps: obtaining the confidence loss, category loss, shape and localization loss of the pedestrian detection box; and setting the loss function as the sum of the confidence loss, the category loss, and the shape and localization loss.

[0010] In one implementation of the first aspect, training the pedestrian detection neural network based on the pedestrian detection bounding boxes using the positive sample selection strategy and the loss function includes the following steps: obtaining positive sample pre-selected boxes for all pedestrian detection bounding boxes using the positive sample selection strategy; training the pedestrian detection neural network based on the positive sample pre-selected boxes until the loss function of the trained pedestrian detection neural network meets a preset requirement.

[0011] In one implementation of the first aspect, pruning the trained pedestrian detection neural network includes the following steps: randomly selecting the number of pruning channels for each convolutional layer based on the trained pedestrian detection neural network; the number of pruning channels is determined by a pruning coefficient; the pruning coefficient is a random small value; sorting the sum of the convolutional kernels for each channel in ascending order and removing the channels with the previously pruned number of channels; updating the batch normalization layer based on the pruned pedestrian detection neural network and obtaining the average precision value of the pruned pedestrian detection neural network; selecting a preset number of different pruning coefficients, obtaining the corresponding average precision values ​​respectively, and calculating the contribution value of each convolution to the average precision value based on the pruning coefficients and the average precision value; determining the pruning weight of each convolution based on each contribution, randomly selecting the number of pruning channels for each convolutional layer based on the pruning weight, and obtaining the average precision value of the pruned pedestrian detection neural network, until the average progress value and computational load meet preset requirements, thus obtaining the pruned pedestrian detection neural network.

[0012] In one implementation of the first aspect, the pedestrian rotating box detection method further includes the following steps: evaluating the trained pedestrian detection neural network model using detection evaluation metrics; the detection evaluation metrics include: recall, accuracy, and comprehensive metrics.

[0013] Secondly, this application provides a pedestrian rotating bounding box detection system, comprising: an acquisition module for acquiring pedestrian detection bounding boxes; a network construction module for constructing a pedestrian detection neural network; a positive sample selection module for setting a positive sample selection strategy for the pedestrian detection neural network; a configuration module for setting a loss function for the pedestrian detection network; a training module for training the pedestrian detection neural network based on the pedestrian detection bounding boxes, using the positive sample selection strategy and the loss function; a pruning module for pruning the trained pedestrian detection neural network; and a testing module for testing the pedestrian detection neural network before and after pruning.

[0014] Finally, this application provides a pedestrian rotation frame detection device, comprising: a processor and a memory. The memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the pedestrian rotation frame detection device performs the pedestrian rotation frame detection method.

[0015] As described above, the pedestrian rotating frame detection method, system, medium, and apparatus of the present invention have the following beneficial effects:

[0016] (1) The model proposed in this application can complete end-to-end detection in one forward propagation and output the position of the rotating box, which can accurately fit the detection target; and the rotating rectangle description method in this application can obtain the box that is closest to the human body. The model output of each channel has little difference in scale, which can achieve better quantization effect on the terminal device and reduce the accuracy loss caused by quantization.

[0017] (2) The pedestrian rotation box detection method of this application can reduce the amount of computation by 40% while ensuring the performance of the model, thereby improving the accuracy of the model and having a smaller amount of computation.

[0018] (3) The pedestrian rotation box detection model of this application has a simple structure and only outputs a feature map of one scale. Because of the adoption of the CBAM module and the reparameter method, features of different scales are well integrated on the output feature map. A feature map of one scale can meet the detection needs in retail scenarios. Attached Figure Description

[0019] Figure 1 The diagram shown is a flowchart of one embodiment of the pedestrian rotation frame detection method of the present invention.

[0020] Figure 2 The diagram shows an implementation of the pedestrian rotation frame detection method of the present invention in an application scenario.

[0021] Figure 3A The diagram shown is a flowchart of step S12 in the pedestrian rotating frame detection method of the present invention.

[0022] Figure 3B The diagram shows a schematic of the network structure constructed in the pedestrian rotating frame detection method of the present invention.

[0023] Figure 3C The diagram shown is a schematic diagram of reparameter training inference in one embodiment of the pedestrian rotation frame detection method of the present invention.

[0024] Figure 3D The diagram shows a schematic of the CBAM structure in one embodiment of the pedestrian rotating frame detection method of the present invention.

[0025] Figure 3E The diagram shown illustrates the rotation frame description in one embodiment of the pedestrian rotation frame detection method of the present invention.

[0026] Figure 4 The diagram shown is a flowchart of step S13 in the pedestrian rotating frame detection method of the present invention.

[0027] Figure 5 The diagram shown is a flowchart of step S14 in the pedestrian rotating frame detection method of the present invention.

[0028] Figure 6 The diagram shown is a flowchart of step S15 in the pedestrian rotating frame detection method of the present invention.

[0029] Figure 7A The diagram shown is a flowchart of step S16 in the pedestrian rotating frame detection method of the present invention.

[0030] Figure 7B The diagram shows a flowchart of the pedestrian detection neural network pruning method of the present invention.

[0031] Figure 8 The diagram shown is a schematic representation of the pedestrian rotating frame detection system of the present invention in one embodiment.

[0032] Figure 9 The diagram shown is a schematic representation of the pedestrian rotating frame detection device of the present invention in one embodiment.

[0033] Component designation explanation

[0034] 81 Acquisition Module

[0035] 82 Network Setup Module

[0036] 83 Positive Sample Screening Module

[0037] 84 Configuration Module

[0038] 85 Training Module

[0039] 86 Pruning Module

[0040] 87 Test Module

[0041] 91 processor

[0042] 92 Memory

[0043] Steps S11 to S17 Detailed Implementation

[0044] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0045] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0046] The pedestrian rotation frame detection method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0047] Please see Figure 1 and Figure 2 The figures shown are a flowchart illustrating one embodiment of the pedestrian rotation frame detection method of the present invention and an implementation diagram illustrating the pedestrian rotation frame detection method of the present invention in an application scenario. Figure 1 and Figure 2 As shown in the figure, this embodiment provides a pedestrian rotating frame detection method.

[0048] The pedestrian rotating frame detection method specifically includes the following steps:

[0049] S11, Obtain pedestrian detection bounding boxes.

[0050] Pedestrian detection boxes were obtained based on an open-source dataset.

[0051] In this embodiment, pedestrian detection images and their annotations are obtained from an open-source dataset on the Internet. The open-source dataset refers to a pedestrian rotation box dataset. This dataset can be divided into a training dataset and a test dataset. The training dataset is used to train the pedestrian detection network; the test dataset is used to evaluate and test the acquired model.

[0052] The pedestrian detection bounding box can be described as: (x center ,y center ,x1,y1,x2,y2,x3,y3,x4,y4); where, (x center ,y center (x1, y1, x2, y2, x3, y3, x4, y4) are the coordinates of the center point of the pedestrian detection box, and (x1, y1, x2, y2, x3, y3, x4, y4) are the coordinates of the vertices of the quadrilateral.

[0053] S12, Construct the pedestrian detection neural network. (See also...) Figure 3A The diagram shows a flowchart of step S12 in the pedestrian rotating frame detection method of the present invention. Figure 3A As shown, step S12 includes the following steps:

[0054] S121, construct a tiny-yolov3 network and add an attention module to the tiny-yolov3 network for feature fusion. (See also...) Figure 3B ,3C The diagrams shown in 3D are a schematic diagram of the network structure constructed in the pedestrian rotation box detection method of the present invention, a schematic diagram of reparameter training and inference in one embodiment of the pedestrian rotation box detection method of the present invention, and a schematic diagram of the CBAM structure in one embodiment of the pedestrian rotation box detection method of the present invention. Figure 3B , 3C As shown in 3D, in this embodiment, the tiny-yolov3 network is used to process the obtained pedestrian detection boxes to obtain several pedestrian features.

[0055] Specifically, the tiny-yolov3 network processes the pedestrian detection boxes to obtain pedestrian feature matrices at different scales, such as stride8, stride16, and stride32. To reduce the performance overhead caused by transmitting and decoding multiple pedestrian features, an attention module is used to fuse the pedestrian feature matrices at different scales.

[0056] Attention modules are used to introduce attention into neural network models, enabling the model to grasp key information and improve its understanding. This embodiment preferably uses the CBAM module. The CBAM module is a lightweight attention module that combines channel and spatial attention mechanisms.

[0057] In this embodiment, since stride16 is the most common scale in retail scenarios with ordinary lenses, the CBAM module processes pedestrian feature matrices of different scales such as stride8, stride16, and stride32, thereby fully fusing the features of all pedestrian feature matrices of different scales and finally obtaining a pedestrian feature matrix with a scale of stride16, which can effectively improve the efficiency of post-processing.

[0058] Meanwhile, in order to fully recognize image information, reparameter training is performed based on the reparams module, so that it can fully learn scene information during training and merge parameters into ordinary convolution during inference.

[0059] S122, Construct the post-processing module of the network. Obtain the detection pre-selection box of the feature map grid number by setting the decoding method of the network feature map matrix.

[0060] The final output of the network is a matrix with channels of (5 coordinate correlation bits + confidence score + number of categories) and the number of feature map grids.

[0061] In this embodiment, the confidence score and category are calculated in the same way as other detection networks, using the sigmoid function. For the box decoding method, please refer to [link to documentation]. Figure 3E The diagram shows a schematic representation of the rotating frame in one embodiment of the pedestrian rotating frame detection method of the present invention. Figure 3E As shown, points p1 and p2 are the midpoints of the two parallel lines, and their coordinates can be calculated from the first four bits of the channel (x1, y1, x2, y2). Here, the stride corresponds to the step size of the feature map it belongs to, which is 16 in this example.

[0062] The formula for calculating the coordinates of the first four positions of the channel is as follows:

[0063]

[0064] The sine and cosine values ​​of the pedestrian detection box angle can be calculated using the following formula:

[0065]

[0066]

[0067] Assume the length of the decision box h1 in the 5th position of channel is: h = e h1 *stride.

[0068] From the above, we can deduce the coordinates of the four points (x1, y1, x2, y2, x3, y3, x4, y4) of the pedestrian detection box. These coordinates can be represented by the following formula:

[0069]

[0070] Based on the above formula, the calculation methods for the pedestrian detection box confidence score, class confidence score, and box position are set so that the coordinates, class confidence score, and box confidence score output by the model are all calculated using e. x The function is used to represent the values ​​of the channels, and the value range of the channels can be within the same range. This allows each variable to be represented by a value within the same range when the model output is quantized with low precision at the terminal, thereby reducing quantization loss.

[0071] S123, The pedestrian detection neural network is constructed based on the tiny-yolov3 network and the network post-processing module.

[0072] In this embodiment, based on the tiny-yolov3 network and the network post-processing module, the pedestrian fusion feature matrix (5 coordinate correlation bits + confidence score + number of categories) * number of pedestrian feature grids obtained in step S121 is set, and the network output matrix is ​​encoded to obtain a pre-selected box with the number of pedestrian feature grids, thus outputting a matrix with the number of channels being (5 coordinate correlation bits + confidence score + number of categories) * number of pedestrian feature grids. That is, during the training process, this module directly outputs the encoded pre-selected boxes. Simultaneously, a deep neural network is trained based on pedestrian features to obtain a pedestrian detection neural network.

[0073] The network post-processing module encodes the network output matrix to obtain pre-selected bounding boxes of the feature map grid. Similarly, during inference, this module performs NMS (non-maximum suppression) processing on the encoded pre-selected bounding boxes to output the final pedestrian detection boxes.

[0074] The tiny-yolov3 network used in this embodiment can also be replaced by other lightweight networks, such as Mobilenet, EfficientNet, ShuffleNet, and combinations of one or more methods, including but not limited to the above networks.

[0075] S13, Set the positive sample selection strategy for the pedestrian detection neural network. (See also...) Figure 4 The diagram shows a flowchart of step S13 in the pedestrian rotating frame detection method of the present invention. Figure 4 As shown, step S13 includes the following steps:

[0076] In this embodiment, a simplified adaptive training sample selection mechanism (ATSS) is used to automatically select positive and negative samples based on the target's statistical information. Simultaneously, a dynamic IOU threshold is employed, adaptively adjusted during training; that is, the network post-processing module directly outputs the encoded pre-selected boxes, and a positive sample filtering algorithm is used to divide the samples into positive and negative categories for use in calculating the loss function. The specific process is as follows:

[0077] S131, select the predicted bounding box of the center point of the actual labeled box and the surrounding 8 points as the initial screening pre-selection box group.

[0078] Based on all the pre-selected bounding boxes directly output by the network post-processing module, the center point of the ground truth bounding box (i.e., ground truth bounding box) and the predicted bounding boxes of the surrounding 8 points are selected as the initial pre-selected bounding box group. Each ground truth bounding box is paired with several initial pre-selected bounding boxes to form the initial pairing group for that ground truth bounding box.

[0079] S132, Deduplicate the initial screening pre-selected boxes. When a pre-selected box is matched with multiple actual annotation boxes in multiple initial screening pre-selected box groups, the pre-selected box is only retained in the initial screening pre-selected box group with the largest intersection-union ratio (IU) with the actual annotation boxes, and is deleted from other initial screening pre-selected box groups. The IU is the ratio of the intersection to the union of the initial screening pre-selected boxes and the actual annotation boxes.

[0080] In this embodiment, when the initial screening pre-selection box matches with two or more ground truth (GT) boxes, the intersection-union ratio (IUR) of the initial screening pre-selection box and the actual annotation box is calculated, and all IUR ratios are compared. The GT box represented by the largest IUR value is selected for pairing, and the initial selection box is deleted from the pairing initial selection box group of other GT boxes.

[0081] S133, calculate the cross-union ratio (CUP) between the true bounding boxes and the pre-selected boxes that match them, calculate the CUP threshold based on the CUP, and select candidate positive samples with CUP greater than the CUP threshold as the final positive samples.

[0082] In this embodiment, the cross-union ratio (CUP) between the ground truth box and its paired preliminary screening box group is calculated. Then, the average and standard deviation of all CUPs are calculated. The average plus the standard deviation is used as the preset CUP threshold. Finally, samples with CUPs greater than the CUP threshold are selected from all candidate positive samples as the final positive samples.

[0083] Specifically, the Intersection over Union (IoU) between the ground truth bounding box and its paired pre-selected bounding box group is calculated. Then, the average IoU (Avg_iou) and standard deviation (Var_iou) of all IoUs are calculated. The sum of the average IoU and the standard deviation (Var_iou) (i.e., the IoU threshold Iou_th = average IoU Avg_iou + standard deviation Var_iou) is used as the preset IoU threshold Iou_th. Finally, samples with IoU greater than the IoU threshold Iou_th are selected from all candidate positive samples as the final positive samples P. final .

[0084] The formula for calculating the intersection-union ratio (IOU) is as follows:

[0085]

[0086] As mentioned above, the value of IOU ranges from 0 to 1. When two pedestrian detection boxes have no intersection, the IOU is 0; when they completely overlap, the IOU is 1. That is, the smaller the overlap, the closer the IOU is to 0, and the larger the overlap, the closer the IOU is to 1.

[0087] Similarly, the positive sample screening method in this application may employ different strategies such as Yolox's SimOTA and the positive sample screening built into Yolo3 to Yolo5, and is not limited to one or a combination of several, including but not limited to the above methods.

[0088] S14, Set the loss function for the pedestrian detection neural network. (See also...) Figure 5 The diagram shows a flowchart of step S14 in the pedestrian rotating frame detection method of the present invention. Figure 5 As shown, S14 includes the following steps:

[0089] S141, obtain the confidence loss, category loss, shape and localization loss of the pedestrian detection box.

[0090] The confidence loss, category loss, shape loss, and localization loss of the pedestrian detection box are calculated based on the loss function.

[0091] The formula for calculating the confidence loss of the pedestrian detection box is:

[0092]

[0093] Among them, conf proposal λ represents the confidence level of the feature map grid output. conf The weighting factor representing the confidence loss of the positive sample preselection box; conf target Indicated as the same as conf proposal A numerical matrix of size, except for those selected as positive sample prediction boxes P final Except for the grid value of 1, the grid values ​​of the other negative sample preselection boxes are all set to 0.

[0094] The formula for calculating the category loss is:

[0095]

[0096] Among them, class gt Indicates the category of the gt box; λ class The weight factors representing the loss of the ground truth bounding box; class Pfinal Represented as P that matches the gt box final The category loss is set as the cross-entropy loss.

[0097] The formulas for calculating the shape and positioning loss functions are as follows:

[0098]

[0099] Among them, coordinate gt The coordinates of the four points of the rectangle containing the gt box; λ coor The coordinate represents the weight factor of the predicted bounding box for positive samples; Pfinal P represents the P that matches the gt box. final The coordinates of the four points of the rectangle; L KFIOU This function represents the approximate IoU loss of two rectangles. Its advantage is that converting the two rectangles to a Gaussian distribution allows for a direct measurement of the difference in their shapes (including angles), avoiding ambiguities arising from angular periodicity and the definitions of long and short sides; and L MSE (coordinate_center gt ,coordinate_center Pfinal The localization loss of the model is measured by calculating the difference between the center points of the two rectangles.

[0100] S142, set the loss function as the sum of the confidence loss, the category loss, and the shape and location losses.

[0101] Based on the loss functions in step S141, calculate the sum of the confidence loss, the category loss, and the shape and location losses.

[0102] The formula for calculating the loss function is as follows:

[0103]

[0104] Among them, conf proposal λ represents the confidence level of the feature map grid output. conf The weighting factor representing the confidence loss of the positive sample preselection box; conft arget Indicated as the same as conf predict Numerical matrix of size; class gt Indicates the category of the gt box; λ class The weight factors representing the loss of the ground truth bounding box; class Pfinal Represented as P that matches the gt box final The category loss is set as cross-entropy loss; coordinate gt The coordinates of the four points of the rectangle containing the gt box; λ coor The coordinate P represents the weight factor of the predicted bounding box for the positive sample. final P represents the P that matches the gt box. final The coordinates of the four points of the rectangle; L KFIOU This represents a function for calculating the approximate IOU loss between two rectangles.

[0105] S15, based on the pedestrian detection box, the pedestrian detection neural network is trained using the positive sample selection strategy and the loss function until the loss function of the trained pedestrian detection neural network meets the preset requirements. (See also...) Figure 6 The diagram shown is a flowchart of step S15 in the pedestrian rotating frame detection method of the present invention.

[0106] In this embodiment, the image from S11 is input into the model in S12 and post-processed to obtain pre-selected boxes. The pre-selected boxes are then divided using a positive sample filtering strategy in S13. The loss is calculated using the loss function in S14, and the loss is backpropagated to update the model. This process is repeated until the loss function no longer decreases, and then the weight file is obtained, resulting in the trained pedestrian detection neural network model. This model has been optimized to achieve high accuracy.

[0107] S16, prune the trained pedestrian detection neural network. See also... Figure 7A and 7B The diagrams shown are schematic flowcharts of S16 in the pedestrian rotating frame detection method of the present invention and the flowchart of the row detection neural network pruning method of the present invention. Figure 7A and 7BAs shown, step S16 includes the following steps:

[0108] S161, randomly select the number of clipped channels for each convolutional layer based on the trained pedestrian detection neural network. The number of clipped channels is determined by a clipping coefficient; the clipping coefficient is a random small value.

[0109] In this embodiment, the pedestrian detection neural network model trained in step S152 is subjected to channel clipping.

[0110] Specifically, assume that the number of channels clipped for each convolutional layer is N1. Here, N1 is determined by the clipping factor k.

[0111] The formula for calculating the number of clipped channels is: Clipped Channels N1 = Clipping Factor k * Total Channels M. The clipping factor k is a randomly selected decimal within the range (0, 0.5). The k value for each convolution is randomly selected within this range, resulting in a set of k and N values. It's important to note that the 3x3 and 1x1 convolutions with reparameterized parameters are considered the same convolution and share the same k value.

[0112] S162, sort the sum of the convolution kernels for each channel in ascending order, and remove the channels whose channel count has been pruned as mentioned above.

[0113] In this embodiment, based on the series of clipped channel numbers obtained in the aforementioned steps, the convolution kernels of each channel are added together and arranged in ascending order, and then the channels of the first N1 convolutions are deleted.

[0114] S163, update the batch normalization layer based on the cropped pedestrian detection neural network, and obtain the average accuracy value of the cropped pedestrian detection neural network.

[0115] In this embodiment, the pruned pedestrian detection neural network performs forward inference on the test dataset in the pedestrian detection dataset to update the batch normalization layer; then, it tests the test dataset to obtain the mean average precision (mAP) of the pruned pedestrian detection neural network.

[0116] The normalization layer is also called the BN layer. The BN layer refers to the process of normalizing the data before it passes through the next layer, so that the mean is 0 and the variance is 1.

[0117] The forward inference described above allows the new model to adjust the Batch Normalization (BN) layers to fit the test dataset, and ensures that the model's performance during evaluation is independent of the BN layers' influence. Therefore, this method of pruned pedestrian detection is more effective at selecting networks that can truly extract pedestrian features.

[0118] S164, select a preset number of different clipping coefficients, each group of clipping coefficients corresponds to a clipping model, obtain the average accuracy value corresponding to the preset number of models respectively, and calculate the contribution value of each convolution to the average accuracy value based on the clipping coefficients and the average accuracy value.

[0119] In this embodiment, a certain number of X different pruning coefficients are selected, and channels are pruned respectively. Then, the convolutional channels are removed based on these pruned channel values. The pruned networks are then tested on the test dataset, and the pruned mAP value is obtained. The k-value and mAP are recorded for each iteration. Each convolution has X k-values, and the correlation between these X k-values ​​and mAP is calculated to obtain the contribution R of the convolution to mAP.

[0120] Specifically, 20 different cutting factors are selected and explained. 20 different cutting factors k1-k are selected. 20 Channels are pruned according to different pruning factors. Then, the convolutional channels are removed based on these pruned channel values. The pruned networks are then tested on a test dataset to obtain the pruned mAP value. Next, each pruning factor k and its corresponding mAP are recorded. At this point, each convolution has 20 pruning factor k values. The contribution of the convolution to the mAP, R, is obtained by calculating the correlation between these 20 k values ​​and mAP.

[0121] S165, search for a pedestrian detection neural network that meets preset requirements based on contribution. Each search determines the number of pruned channels for each convolution based on its contribution and obtains the average accuracy (mAP) of the pruned pedestrian detection neural network. Repeated searches continue until a pruned network meets the preset requirements for both mAP and computational complexity, thus obtaining the pruned pedestrian detection neural network.

[0122] In this embodiment, each search redetermines the number of pruning channels N2 for each convolution based on its contribution R, and then prunes the convolutions according to N2. This results in more convolutions with smaller contributions being pruned, and fewer convolutions with larger contributions being pruned, making it easier to find networks that meet the requirements. The number of pruning channels is determined by the pruning coefficient and the contribution value.

[0123] Specifically, the pruning factor k is re-determined based on each contribution level R. a Next, we obtain the number of clipped channels N2 for each convolution. At this point, the clipping coefficient k aThe value is obtained by multiplying a randomly selected decimal within the range (0, 0.3) by (1 - contribution). Then, the pedestrian detection neural network is pruned based on the new pruned channel count N2, and tested on the test dataset to obtain the mAP under the new pruned channel count N2. This pruning and testing process is repeated until a network with 60% of the computational cost of the model and 70% of the mAP of the pedestrian detection neural network model is obtained. In this example, repeating this process 30 times is sufficient to find a suitable model structure and pruned model weights.

[0124] S17, Load the cropped convolutional weights to train the network to restore accuracy.

[0125] In this embodiment, the pedestrian detection neural network model obtained in step S16 is used. The training dataset is input into the final pedestrian detection neural network model, and its loss function is calculated through a positive sample selection strategy for training. The training is iterated until the loss function no longer decreases, and then the weights of the pedestrian detection neural network model after accuracy recovery are obtained.

[0126] Next, test the effect of cropping the network.

[0127] The detection performance of the pruning model is statistically analyzed using the test dataset. Optimal model accuracy metrics can be evaluated using: recall (R), precision (P), and the F-measure. Higher values ​​for these metrics indicate better model detection performance, while the F-measure balances recall and precision to assess overall model effectiveness.

[0128] The evaluation indicators are calculated as follows:

[0129] Recall rate = Total number of correctly detected targets / Total number of targets

[0130] Accuracy = Total number of correctly detected targets / Total number of detected targets

[0131] The formula for calculating the comprehensive index is as follows:

[0132]

[0133] Here, α represents the importance of recall and precision in the evaluation. When α = 1, recall and precision are equally important. In this example, the test metrics of the pruned network can be completely restored to those of the network before pruning.

[0134] The pedestrian rotating bounding box detection method presented in this application provides a suitable quantized rotating rectangular bounding box description method for retail customer flow detection applications. It employs a lightweight deep learning network design with a common convolutional structure, utilizes a high-parameter training method and a proposed efficient pruning method with Batch Normalization (BN) layers to minimize model computation, making the final model easy to deploy. It maintains the original accuracy while reducing the computational cost of the original model by 40%. This application improves the computing power of the terminal device by reducing the computational cost of the detection model without sacrificing coverage. Simultaneously, it reduces the large quantization error caused by the difference in length and angle between the same feature map output during low-precision quantization at the terminal; it also significantly reduces the performance overhead of the terminal device's computation.

[0135] The protection scope of the pedestrian rotating frame detection method described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the protection scope of this application.

[0136] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following... Figure 1 The pedestrian rotation frame detection method.

[0137] At any possible level of technical detail, this application can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0138] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0139] The computer-readable program described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards these instructions to a computer-readable storage medium in the respective computing / processing device. The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as "C" or similar programming languages. Computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of this application.

[0140] This application also provides a pedestrian rotation frame detection system, which can implement the pedestrian rotation frame detection method described in this application. However, the implementation device of the pedestrian rotation frame detection method described in this application includes, but is not limited to, the structure of the pedestrian rotation frame detection system listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0141] The pedestrian rotating frame detection system provided in this embodiment will be described in detail below with reference to the illustrations.

[0142] This embodiment provides a pedestrian rotating frame detection system, including:

[0143] Please see Figure 8The diagram shows a schematic representation of the pedestrian rotating frame detection system of the present invention in one embodiment. Figure 8 As shown, the pedestrian rotating frame detection system includes: an acquisition module 81, a network construction module 82, a positive sample screening module 83, a configuration module 84, a training module 85, a pruning module 86, and a testing module 87.

[0144] The acquisition module 81 is used to acquire pedestrian detection boxes.

[0145] In this embodiment, pedestrian detection boxes are obtained based on an open-source dataset.

[0146] Specifically, pedestrian detection images and their annotations can be obtained from open-source datasets on the internet. These open-source datasets refer to pedestrian rotation box datasets. These datasets can be divided into training and testing datasets. The training dataset is used to train the pedestrian detection network; the testing dataset is used to evaluate and test the acquired model.

[0147] The pedestrian detection bounding box can be described as: (x center ,y center ,x1,y1,x2,y2,x3,y3,x4,y4); where, (x center ,y center (x1, y1, x2, y2, x3, y3, x4, y4) are the coordinates of the center point of the pedestrian detection box, and (x1, y1, x2, y2, x3, y3, x4, y4) are the coordinates of the vertices of the quadrilateral.

[0148] The network construction module 82 is connected to the acquisition module 81 and is used to construct a pedestrian detection neural network.

[0149] Construct a tiny-yolov3 network and add an attention module to the tiny-yolov3 network for feature fusion.

[0150] In this embodiment, the tiny-yolov3 network is used to process the obtained pedestrian detection boxes to obtain several pedestrian features.

[0151] Specifically, the tiny-yolov3 network processes the pedestrian detection boxes to obtain pedestrian feature matrices at different scales, such as stride8, stride16, and stride32. To reduce the performance overhead caused by transmitting and decoding multiple pedestrian features, an attention module is used to fuse the pedestrian feature matrices at different scales.

[0152] Attention modules are used to introduce attention into neural network models, enabling the model to grasp key information and improve its understanding. This embodiment preferably uses the CBAM module. The CBAM module is a lightweight attention module that combines channel and spatial attention mechanisms.

[0153] In this embodiment, since stride16 is the most common scale in retail scenarios with ordinary lenses, the CBAM module processes pedestrian feature matrices of different scales such as stride8, stride16, and stride32, thereby fully fusing the features of all pedestrian feature matrices of different scales and finally obtaining a pedestrian feature matrix with a scale of stride16, which can effectively improve the efficiency of post-processing.

[0154] Meanwhile, in order to fully recognize image information, reparameter training is performed based on the reparams module, so that it can fully learn scene information during training and merge parameters into ordinary convolution during inference.

[0155] Next, a post-processing module is constructed. The detection pre-selection box is obtained by setting the decoding method of the network feature map matrix, which determines the number of feature map grids. The final network output is a matrix with channels equal to (5 coordinate correlation bits + confidence score + number of categories) and the number of feature map grids.

[0156] In this embodiment, the confidence score and category are calculated in the same way as other detection networks, and can be obtained through the Sigmoid function.

[0157] Based on relevant calculation formulas, the calculation methods for pedestrian detection box confidence, class confidence, and box position are set so that the coordinates, class confidence, and box confidence output by the model are all calculated using e. x The function is used to represent the values ​​of the channels, and the value range of the channels can be within the same range. This allows each variable to be represented by a value within the same range when the model output is quantized with low precision at the terminal, thereby reducing quantization loss.

[0158] Finally, the pedestrian detection neural network is constructed based on the tiny-yolov3 network and the network post-processing module.

[0159] In this embodiment, based on the tiny-yolov3 network and the network post-processing module, a pedestrian fusion feature matrix (5 coordinate correlation bits + confidence score + number of categories) * number of pedestrian feature grids is set. The network output matrix is ​​encoded to obtain pre-selected boxes for the number of pedestrian feature grids, thus outputting a matrix of (5 coordinate correlation bits + confidence score + number of categories) * number of pedestrian feature grids. That is, during training, this module directly outputs the encoded pre-selected boxes. Simultaneously, a deep neural network is trained based on pedestrian features to obtain a pedestrian detection neural network.

[0160] The network post-processing module encodes the network output matrix to obtain pre-selected bounding boxes of the feature map grid. Similarly, during inference, this module performs NMS (non-maximum suppression) processing on the encoded pre-selected bounding boxes to output the final pedestrian detection boxes.

[0161] The networks used in this embodiment, such as tiny-yolov3, and algorithms such as Mobilenet, EfficientNet, and ShuffleNet, can all be run on processing servers or terminals such as CPUs, NPUs, or GPUs.

[0162] The positive sample screening module 83 is used to set the positive sample screening strategy of the pedestrian detection neural network.

[0163] The positive sample selection strategy for the pedestrian detection neural network is set. In this embodiment, a simplified adaptive sample selection mechanism (ATSS) is used to automatically select positive and negative samples based on the statistical information of the target. Simultaneously, a dynamic IOU threshold is employed, adaptively adjusted during training; that is, the network post-processing module directly outputs the encoded pre-selected boxes, and the positive and negative samples are divided using the positive sample selection algorithm for use in calculating the loss function.

[0164] Specifically, the center point of the ground truth bounding box and the predicted bounding boxes of its eight surrounding points are selected as the initial screening pre-selection box group. Based on all the pre-selection boxes directly output by the network post-processing module, each ground truth bounding box is paired with several predicted bounding boxes of the center point of the ground truth bounding box (i.e., the ground truth bounding box) and its eight surrounding points as the initial screening pre-selection box group. Each ground truth bounding box is paired with several initial screening pre-selection boxes to form the initial selection pairing group for that ground truth bounding box.

[0165] The initial screening pre-selected boxes are deduplicated. When a pre-selected box matches multiple actual labeled boxes in multiple initial screening pre-selected box groups, the pre-selected box is only retained in the initial screening pre-selected box group with the highest intersection-union ratio (Intersection over Union) with the actual labeled boxes, and is deleted from the other initial screening pre-selected box groups. The intersection-union ratio is the ratio of the intersection and union of the initial screening pre-selected boxes and the actual labeled boxes.

[0166] In this embodiment, when the initial screening pre-selection box matches with two or more ground truth (GT) boxes, the intersection-union ratio (IUR) of the initial screening pre-selection box and the actual annotation box is calculated, and all IUR ratios are compared. The GT box represented by the largest IUR value is selected for pairing, and the initial selection box is deleted from the pairing initial selection box group of other GT boxes.

[0167] Calculate the cross-union ratio (CUI) between the true bounding boxes and the pre-selected boxes that match them, calculate the CUI threshold based on the CUI, and select candidate positive samples whose CUI is greater than the CUI threshold as the final positive samples.

[0168] In this embodiment, the cross-union ratio (CUP) between the ground truth box and its paired preliminary screening box group is calculated. Then, the average and standard deviation of all CUPs are calculated. The average plus the standard deviation is used as a preset CUP threshold. Finally, samples with CUPs greater than the CUP threshold are selected from all candidate positive samples as the final positive samples.

[0169] Similarly, the positive sample screening method in this application may employ different strategies such as Yolox's SimOTA and the positive sample screening built into Yolo3 to Yolo5, and is not limited to one or a combination of several, including but not limited to the above methods.

[0170] Configuration module 84 is used to set the loss function of the pedestrian detection network.

[0171] In this embodiment, the confidence loss, category loss, shape and localization loss of the pedestrian detection box are obtained.

[0172] The confidence loss, category loss, and shape and location loss of the pedestrian detection box are calculated separately based on the loss function. The loss function is then set as the sum of the confidence loss, category loss, and shape and location losses.

[0173] The training module 85 is used to train the pedestrian detection neural network based on the pedestrian detection box, using the positive sample screening strategy and the loss function, until the loss function of the trained pedestrian detection neural network meets the preset requirements.

[0174] In this embodiment, images from the training dataset are input into the constructed model and post-processed to obtain pre-selected bounding boxes. A positive sample filtering strategy is used to divide the pre-selected boxes, which are then fed into a loss function for loss calculation. The loss is backpropagated to update the model, iterating until the loss function no longer decreases. Finally, a weight file is obtained, resulting in the trained pedestrian detection neural network model. This model has been optimized to achieve high accuracy.

[0175] The pruning module 86 is used to prune the trained pedestrian detection neural network.

[0176] In this embodiment, the number of clipped channels in each convolutional layer is randomly selected based on the trained pedestrian detection neural network. The number of clipped channels is determined by a clipping coefficient, which is a random small value.

[0177] The sum of the convolution kernels for each channel is sorted in ascending order, and the channels whose number of channels has been clipped as mentioned above are removed.

[0178] The batch normalization layer is updated based on the cropped pedestrian detection neural network, and the average accuracy value of the cropped pedestrian detection neural network is obtained.

[0179] In this embodiment, the pruned pedestrian detection neural network performs forward inference on the test dataset from the pedestrian detection dataset to update the Batch Normalization (BN) layer. Then, it is tested again on the test dataset to obtain the mAP of the pruned pedestrian detection neural network. This forward inference adjusts the BN layer of the new model to fit the test dataset, and ensures that the model's performance during evaluation is independent of the influence of the BN layer. Therefore, this pruned pedestrian detection method is more likely to select a network that can truly extract pedestrian features.

[0180] Select a preset number of different clipping coefficients, each group of clipping coefficients corresponds to a clipping model, obtain the average accuracy value corresponding to the preset number of models, and calculate the contribution value of each convolution to the average accuracy value based on the clipping coefficients and the average accuracy value.

[0181] In this embodiment, a certain number of X different pruning coefficients are selected, and channels are pruned respectively. Then, the convolutional channels are removed based on these pruned channel values. The pruned networks are then tested on the test dataset, and the pruned mAP value is obtained. The k-value and mAP are recorded for each iteration. Each convolution has X k-values, and the correlation between these X k-values ​​and mAP is calculated to obtain the contribution R of the convolution to mAP.

[0182] The search algorithm searches for a pedestrian detection neural network that meets preset requirements based on contribution. Each search determines the number of pruned channels for each convolution based on its contribution and obtains the average accuracy (mAP) of the pruned neural network. This process is repeated multiple times until a pruned network meets both the preset requirements for mAP and computational complexity. This results in the pruned pedestrian detection neural network.

[0183] Test module 87 is used to test the pedestrian detection neural network before and after pruning.

[0184] In this embodiment, the cropped convolutional weights are loaded to train the network to restore accuracy. The training dataset is input into the final pedestrian detection neural network model, and its loss function is calculated using a positive sample selection strategy for training. The training is iterated until the loss function no longer decreases, and then the weights of the pedestrian detection neural network model after accuracy restoration are obtained.

[0185] Then, the detection performance of the pruning model is statistically analyzed using the test dataset. The optimal model accuracy evaluation metrics can be achieved using: recall (R), precision (P), and a combined metric (F) to assess model performance.

[0186] The pedestrian rotation box detection model enables the construction of a pedestrian rotation box detection system, which can improve the computing power of terminal devices in retail customer flow detection systems while reducing the computational load of the detection model without sacrificing coverage. Simultaneously, it reduces quantization errors and improves the computing performance and lifespan of the terminal devices.

[0187] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, module x can be a separate processing element, or it can be integrated into a chip within the system. Alternatively, it can be stored as program code in the system's memory, and its function can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0188] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SoC).

[0189] Please see Figure 9 The diagram shows a schematic representation of the pedestrian rotating frame detection device of the present invention in one embodiment. Figure 9 As shown, this embodiment provides a pedestrian rotation frame detection device, which includes: a processor 91 and a memory 92; the memory 92 is used to store computer programs; the processor 91 is connected to the memory 92 and is used to execute the computer programs stored in the memory 92, so that the pedestrian rotation frame detection device performs each step of the pedestrian rotation frame detection method as described above.

[0190] Preferably, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0191] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0192] In summary, the pedestrian rotating frame detection method, system, medium, and device provided in this application have the following beneficial effects:

[0193] The proposed model can complete end-to-end detection in a single forward propagation, outputting the position of the rotated bounding box, which accurately fits the detected target. Furthermore, the rotated rectangle description method used in this application yields a bounding box that most closely resembles the human body. The model outputs a relatively uniform scale across different channels, resulting in better quantization performance on terminal devices and reducing accuracy loss caused by quantization. Simultaneously, it reduces computation by 40% while maintaining model performance, improving model accuracy with minimal computational cost. The pedestrian rotated bounding box detection model has a simple structure, outputting a feature map at only one scale, which allows for excellent fusion of features at different scales on the output feature map, meeting the detection needs in retail scenarios.

[0194] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting pedestrians using a rotating frame, characterized in that, Includes the following steps: Obtain pedestrian detection bounding boxes; Constructing a pedestrian detection neural network includes: constructing a tiny-yolov3 network and adding an attention module to the tiny-yolov3 network for feature fusion; setting a network post-processing module; and constructing the pedestrian detection neural network based on the tiny-yolov3 network and the network post-processing module. Setting a positive sample selection strategy for the pedestrian detection neural network; including: The predicted bounding boxes of the center point of the actual bounding box and the surrounding 8 points are selected as the initial screening pre-selection box group of the actual bounding box; When a preselected box is matched with multiple actual annotation boxes in multiple initial screening preselected box groups, the preselected box is only retained in the group with the largest intersection-union ratio with the actual annotation boxes, and the preselected box is deleted in other initial screening preselected box groups. Calculate the intersection-union ratio (CUI) between the true bounding boxes and the pre-selected boxes that match them, calculate the CUI threshold based on the CUI, and select candidate positive samples whose CUI is greater than the CUI threshold as the final positive samples; The loss function of the pedestrian detection neural network is set as the sum of confidence loss, category loss, shape loss, and localization loss; wherein the shape loss and localization loss are approximated by converting the rotated box to a Gaussian distribution and calculating the similarity of the Gaussian distribution. Based on the pedestrian detection bounding box, the pedestrian detection neural network is trained using the positive sample selection strategy and the loss function. Pruning the trained pedestrian detection neural network; including: The number of clipped channels in each convolutional layer is randomly selected based on the trained pedestrian detection neural network; the number of clipped channels is determined by a clipping coefficient; the clipping coefficient is a random small value. The sum of the convolution kernels for each channel is sorted in ascending order, and the channels whose channel count has been pruned as mentioned above are removed. The batch normalization layer is updated based on the cropped pedestrian detection neural network, and the average accuracy value of the cropped pedestrian detection neural network is obtained. Select a preset number of different clipping coefficients, obtain the corresponding average precision values, and calculate the contribution of each convolution to the average precision value based on the clipping coefficients and the average precision values. Based on each contribution, determine the pruning weight of each convolution, select the number of pruning channels for each convolution layer based on the pruning weight, obtain the average accuracy value of the pruned pedestrian detection neural network, and continue until the average progress value and computational load meet the preset requirements to obtain the pruned pedestrian detection neural network. Based on the pedestrian detection bounding box, the pruned pedestrian detection neural network is retrained using the positive sample screening strategy and the loss function, so as to perform pedestrian detection based on the retrained pedestrian detection neural network.

2. The pedestrian rotating frame detection method according to claim 1, characterized in that, Obtaining pedestrian detection bounding boxes includes the following steps: Pedestrian detection boxes were obtained based on an open-source dataset.

3. The pedestrian rotating frame detection method according to claim 1, characterized in that, Training the pedestrian detection neural network based on the pedestrian detection bounding box, using the positive sample selection strategy and the loss function, includes the following steps: The positive sample screening strategy described above is used to obtain the positive sample pre-selection boxes for all pedestrian detection boxes; The pedestrian detection neural network is trained based on the positive sample preselection box until the loss function of the trained pedestrian detection neural network meets the preset requirements.

4. The pedestrian rotating frame detection method according to claim 1, characterized in that, It also includes the following steps: The trained pedestrian detection neural network model is evaluated using detection and evaluation metrics, including recall, accuracy, and a comprehensive metric.

5. A pedestrian rotating frame detection system, characterized in that, include: The acquisition module is used to acquire pedestrian detection boxes; A network construction module is used to build a pedestrian detection neural network; including: building a tiny-yolov3 network and adding an attention module to the tiny-yolov3 network for feature fusion; setting a network post-processing module; and building the pedestrian detection neural network based on the tiny-yolov3 network and the network post-processing module. The positive sample screening module is used to set the positive sample screening strategy of the pedestrian detection neural network. This includes: selecting the predicted bounding boxes of the center point of the ground truth bounding box and its eight surrounding points as the initial screening pre-selection box group; when a pre-selection box is matched with multiple ground truth bounding boxes in multiple initial screening pre-selection box groups, the pre-selection box is only retained in the group with the highest intersection-union ratio (IU) with the ground truth bounding boxes, and is deleted from other initial screening pre-selection box groups; calculating the IU between the ground truth bounding box and the initial screening pre-selection box group it matches, calculating an IU threshold based on the IU, and selecting candidate positive samples with an IU greater than the IU threshold as the final positive samples. The configuration module is used to set the loss function of the pedestrian detection network, wherein the loss function is the sum of confidence loss, category loss, shape loss, and localization loss; The training module is used to train the pedestrian detection neural network based on the pedestrian detection bounding box, using the positive sample selection strategy and the loss function; A pruning module is used to prune a trained pedestrian detection neural network. This includes: randomly selecting the number of pruning channels for each convolutional layer based on the trained pedestrian detection neural network; the number of pruning channels is determined by a pruning coefficient, which is a random small value; sorting the sum of the convolutional kernels for each channel in ascending order and removing channels whose pruning count has already been reduced; updating the batch normalization layer based on the pruned pedestrian detection neural network and obtaining the average precision value of the pruned pedestrian detection neural network; selecting a preset number of different pruning coefficients, obtaining the corresponding average precision values ​​for each, and calculating the contribution of each convolution to the average precision value based on the pruning coefficients and the average precision values; determining the pruning weight of each convolution based on each contribution; selecting the number of pruning channels for each convolutional layer based on the pruning weights; obtaining the average precision value of the pruned pedestrian detection neural network; and continuing until the average progress value and computational load meet preset requirements to obtain the pruned pedestrian detection neural network. The testing module is used to test the pedestrian detection neural network before and after pruning.

6. A pedestrian rotating frame detection device, characterized in that, include: Processor and memory; The memory is used to store computer programs; The processor is connected to the memory and is used to execute the computer program stored in the memory so that the pedestrian rotating frame detection device performs the pedestrian rotating frame detection method according to any one of claims 1 to 4.

Images