Human body detection angle encoding / image training methods, systems, terminals and media

Abstract

This invention provides a human detection angle encoding / image training method, system, terminal, and medium. The human detection angle encoding method includes: acquiring a human detection bounding box based on a top-view perspective; sorting the vertices of the human detection bounding box in a preset order; after the human detection bounding box rotates, encoding the human detection bounding box based on the sorted vertices to obtain an encoding result; the encoding result includes the size of the human detection bounding box, the angle segment decomposed based on the rotation angle of the human detection bounding box, and the angle segment offset. The human detection angle encoding method / system of this invention reduces the amount of model output data. Models using this encoding method can reduce bandwidth consumption throughout the inference process, improving application performance. It is particularly suitable for application on front-end devices.

Description

Technical Field

[0001] This invention belongs to the field of human body detection technology, and relates to a method, particularly a human body detection angle encoding / image training method, system, terminal and medium. Background Technology

[0002] Object detection, especially human detection, is an important research area in computer vision. As a detection target, the human body is more complex than ordinary objects. The human body is a non-rigid object, exhibiting a wide variety of postures, and the diversity of clothing and complex scenes also present significant challenges to solving this problem. Currently, the mainstream approach uses deep learning-based methods, abandoning manually designed features and instead designing complex network structures and employing appropriate training methods.

[0003] Human detection technology has wide applications in commercial scenarios and has significant commercial value. Surveillance cameras are typically used as data acquisition points, and oblique illumination human detection algorithms are employed in these cases.

[0004] However, in actual project engineering, the installation height and angle of surveillance cameras often cannot meet business requirements, so top-view installation is often chosen. This results in human images appearing at 360° different angles in the image. Since the mainstream method is horizontal bounding box detection, it lacks angular information. The model does not learn human angles during the training phase, therefore the localization of human detection boxes from the top view is inaccurate, and it lacks the human angle detection results required for business applications. Currently, methods for detecting rotating objects are mainly used for target detection in remote sensing images and text box detection. These targets are rigid objects, which are very different from human targets. Therefore, a specific model structure for top-view human detection is needed.

[0005] In practical engineering applications, intelligent front-end devices are used to simultaneously perform video acquisition and data analysis. These devices possess chips that support AI inference, but due to cost and power consumption limitations, their computing power and performance are far inferior to GPU inference servers. Therefore, AI applications deployed on front-end devices must balance model accuracy and performance. Furthermore, considering that model inference results need to be copied from the AI ​​chip / CPU to main memory, the model's output cannot consume excessive bandwidth.

[0006] AI applications on front-end devices must be designed with computing power requirements in mind, while reducing the amount of output data of the model and reducing the bandwidth usage between the AI ​​inference chip / CPU and main memory.

[0007] In top-view human detection applications, human targets include coordinates, width and height, target category, and rotation angle. Current methods produce a very large data dimension in the model output. After inference is performed on the front-end device, copying the model output from the NPU / CPU to main memory takes a considerable amount of time. This is detrimental to deploying top-view human detection applications on front-end devices.

[0008] Therefore, how to provide a human body detection angle encoding / image training method, system, terminal and medium to solve the shortcomings of existing technologies, such as the difficulty of deploying top-view human body detection applications on intelligent front-end devices, has become a technical problem that urgently needs to be solved by the technical personnel in this field. Summary of the Invention

[0009] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a human body detection angle encoding / image training method, system, terminal and medium to solve the problem of difficulty in deploying top-view human body detection applications on intelligent front-end devices.

[0010] To achieve the above and other related objectives, the present invention provides a human body detection angle encoding method, comprising: acquiring a human body detection box based on a top viewpoint; sorting the vertices of the human body detection box in a preset order; after the human body detection box is rotated, encoding the human body detection box according to the sorted vertices of the human body detection box to obtain an encoding result; the encoding result includes the size of the human body detection box, the angle segment decomposed based on the rotation angle of the human body detection box, and the angle segment offset.

[0011] In one embodiment of the present invention, the preset order of the vertices of the human body detection frame is from the top left vertex to the top right vertex, from the top right vertex to the bottom left vertex, and from the bottom right vertex to the bottom left vertex.

[0012] In one embodiment of the present invention, the step of encoding the human detection box according to the sorted vertices of the human detection box to obtain the encoding result includes: calculating the width of the human detection box based on the distance between the coordinates of the top left and top right vertices, and calculating the length of the human detection box based on the distance between the coordinates of the top left and bottom left vertices; calculating the horizontal length and vertical length of the top left and bottom left vertices based on the coordinates of the top left and bottom left vertices, and deriving the rotation angle of the human detection box based on the calculated horizontal and vertical lengths; retaining the calculated rotation angle when half of the sum of the horizontal coordinates of the top left and top right vertices is less than the center coordinates of the human detection box; correcting the rotation angle when half of the sum of the horizontal coordinates of the top left and top right vertices is greater than or equal to the center coordinates of the human detection box; the corrected rotation angle is equal to 360 degrees minus the calculated rotation angle; and decomposing the rotation angle of the human detection box into angle segments and angle segment offsets.

[0013] In one embodiment of the present invention, the rotation angle is converted from a radian value to an angle value; wherein, the integer value of the angle value divided by 10 is the angle segment, and the units digit of the angle value is the angle segment offset.

[0014] Another aspect of the present invention provides an image training method based on a human detection angle encoding method. The image training method includes: receiving training images based on a top-view perspective; extracting features from the top-view training images to output feature maps at different scales; fusing the feature maps of different sizes to form stitched feature maps of different scales; predicting the results of the stitched feature maps of different sizes and decoding the prediction results to obtain decoding results; the decoding results include target heatmaps at different scales, the size of the human detection box, target classification, angle segments and angle segment offsets of the rotation angle of the human detection box; obtaining the encoding results of the training images based on the human detection angle encoding method, and calculating losses for the target heatmaps, the size of the human detection box, the target classification, and the angle segments and angle segment offsets of the rotation angle of the human detection box at different scales.

[0015] In one embodiment of the present invention, the prediction result includes a large-scale prediction result, a medium-scale prediction result, and a small-scale prediction result; the number of channels for each pixel in the large-scale prediction result, the medium-scale prediction result, and the small-scale prediction result is 51; wherein, the first channel is the confidence score, the second and third channels are the width and height of the human detection box, the fourth and fifth channels are the target classification score, the sixth to the forty-first channels are the angle segment score of the rotation angle of the human detection box, and the forty-second to the fifty-first channels are the angle segment offset score of the rotation angle.

[0016] In one embodiment of the present invention, the steps of predicting the results of spliced ​​feature maps of different sizes and decoding the prediction results to obtain the decoding results include: performing normalization calculation and non-maximum suppression calculation on the first channel; filtering the maxima using a detection confidence threshold, setting the coordinate index of the remaining maxima as the coordinates of the predicted target, and forming a set of predicted targets; for each coordinate in the set of predicted targets, obtaining the values ​​corresponding to the second and third channels to predict the width and height of the human detection box; for each coordinate in the set of predicted targets, obtaining the score values ​​corresponding to the fourth and fifth channels, and setting the index of the maximum score value as the predicted category of the target; for each coordinate in the set of predicted targets, obtaining the angle values ​​corresponding to the sixth to forty-first channels, and setting the index of the maximum angle value as the predicted angle segment of the rotation angle of the human detection box; for each coordinate in the set of predicted targets, obtaining the angle segment offset values ​​corresponding to the forty-second to fifty-first channels, and setting the index of the maximum angle segment offset value as the predicted angle segment offset of the rotation angle of the human detection box.

[0017] In one embodiment of the present invention, the steps of obtaining the encoding results of the training image based on the human detection angle encoding method, and calculating the loss of the target heatmap, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box and the angle segment offset at different scales, include: calculating the loss of the target heatmap based on the coordinates of the target in the training image and the coordinates of the predicted target; calculating the human detection box size loss based on the size of the human detection box in the training image and the predicted width and height of the human detection box; calculating the loss of the target category based on the category of the training image and the predicted category; calculating the loss of the angle segment based on the angle segment of the training image and the predicted angle segment; and calculating the loss of the angle segment offset based on the angle segment offset of the training image and the predicted angle segment offset.

[0018] In another aspect, the present invention provides a human body detection angle encoding system, comprising: an acquisition module for acquiring a human body detection box based on a top-view perspective; a sorting module for sorting the vertices of the human body detection box in a preset order; and an encoding module for encoding the human body detection box according to the sorted vertices after the human body detection box has rotated, so as to obtain an encoding result; the encoding result includes the size of the human body detection box, the angle segment decomposed based on the rotation angle of the human body detection box, and the angle segment offset.

[0019] In another aspect, this invention provides an image training system based on a human detection angle encoding system. The image training system includes: a feature extraction module for receiving training images based on a top-view perspective, extracting features from the top-view training images, and outputting feature maps at different scales; a feature fusion module for fusing features from feature maps of different sizes to form stitched feature maps of different scales; a prediction module for predicting the results of the stitched feature maps of different sizes and decoding the prediction results to obtain decoding results; the decoding results include target heatmaps at different scales, the size of the human detection box, target classification, and the angle segments and angle segment offsets of the rotation angle of the human detection box; and a loss calculation module for obtaining the encoding results of the training images based on the human detection angle encoding method, and calculating the loss for the target heatmaps, the size of the human detection box, target classification, and the angle segments and angle segment offsets of the rotation angle of the human detection box at different scales.

[0020] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human body detection angle encoding method and / or image training method.

[0021] The final aspect of the present invention provides a terminal, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the terminal to perform the human body detection angle encoding method and / or the image training method.

[0022] As described above, the human body detection angle encoding / image training method, system, terminal, and medium of the present invention have the following beneficial effects:

[0023] First, the human body detection angle encoding method / system described in this invention reduces the amount of model output data. Models using this encoding method can reduce bandwidth consumption throughout the inference process, thus improving application performance. It is particularly suitable for application on front-end devices.

[0024] Second, the image training method / system based on the human body detection angle encoding method described in this invention is adapted to the above-mentioned human target encoding method / system, and the model structure is a lightweight design, suitable for deployment on front-end devices. Attached Figure Description

[0025] Figure 1 The diagram shown is a flowchart of one embodiment of the human body detection angle encoding method of the present invention.

[0026] Figure 2 The diagram shown is an example of the top-view human detection frame of the present invention.

[0027] Figure 3The diagram shows the vertex sequence of the top-view human detection box of the present invention.

[0028] Figure 4 The diagram shown is a flowchart of S13 in the human body detection angle encoding method of the present invention.

[0029] Figure 5 The diagram shown is a flowchart of an embodiment of the image training method based on the human body detection angle encoding method of the present invention.

[0030] Figure 6 The diagram shown is a schematic of the pre-stored Backbone structure of the present invention.

[0031] Figure 7 The diagram shown is a schematic of the CSP-output channel number labeling convolution module of the present invention.

[0032] Figure 8 The diagram shown is a schematic representation of the spatial pyramid pooling structure of this invention.

[0033] Figure 9 The diagram shown is a schematic diagram of the pre-stored feature fusion structure of the present invention.

[0034] Figure 10 The diagram shown is an example of the encoded value of the target in this invention.

[0035] Figure 11A The diagram shown is a schematic diagram of the principle structure of a human body detection angle encoding system in one embodiment.

[0036] Figure 11B The diagram shown is a schematic representation of the principle structure of an image training system in one embodiment.

[0037] Component designation explanation

[0038] Detailed Implementation

[0039] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0040] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0041] Example 1

[0042] This embodiment provides a human body detection angle encoding method, including:

[0043] Obtain the human detection bounding box based on the top view;

[0044] The vertices of the human body detection frame are sorted in a preset order;

[0045] After the human body detection box rotates, the human body detection box is encoded according to the sorted vertices of the human body detection box to obtain the encoding result; the encoding result includes the size of the human body detection box, the angle segment decomposed based on the rotation angle of the human body detection box, and the angle segment offset.

[0046] The human body detection angle encoding method provided in this embodiment will be described in detail below with reference to the illustrations. Please refer to... Figure 1 The diagram shows a flowchart of a human body detection angle encoding method in one embodiment. Figure 1 As shown, the human body detection angle encoding method specifically includes the following steps:

[0047] S11, Obtain the human detection bounding box based on the top view. See also... Figure 2 The image shown is an example of a human detection bounding box based on a top-view perspective.

[0048] S12, sort the vertices of the human body detection box in a preset order.

[0049] Please see Figure 3 This is displayed as a vertex sequence diagram of the human detection bounding box based on a top-view perspective. For example... Figure 3 As shown, T represents the human body, C is the center of the human body target bounding box, P1 is the upper left corner of the human body detection box, P2 is the upper right corner, P3 is the lower right corner, and P4 is the lower left corner.

[0050] The preset order of the vertices of the human body detection frame is from the top left vertex P1 to the top right vertex P2, from the top right vertex P2 to the bottom left vertex P3, and from the bottom right vertex P3 to the bottom left vertex P4.

[0051] In the top view, the human body detection box is selected around the image center C. The human body is located at any position in the image, and P1 is fixed at the upper left corner of the human body. The four vertices of the human body target box are rearranged according to P1→P2→P3→P4.

[0052] S13, after the human detection box rotates, the human detection box is encoded according to the sorted vertices of the human detection box to obtain the encoding result; the encoding result includes the size of the human detection box, the angle segment decomposed based on the rotation angle of the human detection box, and the angle segment offset. (See [link to relevant documentation]). Figure 4 The flowchart shown is for S13. Figure 4 As shown, step S13 includes the following steps:

[0053] S131, calculate the width of the human detection box based on the distance between the coordinates of the top left and top right vertices, and calculate the length of the human detection box based on the distance between the coordinates of the top left and bottom left vertices.

[0054] In this embodiment, the length and width of the detection frame are calculated using the distance formula between two points.

[0055] S132, based on the coordinates of the top left and bottom left vertices, calculate the horizontal length and vertical length of the top left and bottom left vertices.

[0056] Specifically, the horizontal length vec of the top-left vertex and the bottom-left vertex x Equal to the x-coordinate P1 of the top left vertex x Subtract the x-coordinate of the bottom left vertex P4 x , i.e., vec x =P1 x -P4 x .

[0057] The vertical length vec of the top left vertex and the bottom left vertex y Equal to the ordinate P4 of the lower left corner vertex y Subtract the ordinate P1 of the top left vertex y , i.e., vec y =P4 y -P1 y .

[0058] S133, calculate the rotation angle θ of the human body detection frame based on the calculated horizontal and vertical lengths.

[0059] Specifically,

[0060] S134, when the x-coordinate of the top left vertex is P1 x x-coordinate P2 of the top right vertex x Half of the sum is less than the center coordinate C of the human detection box. x When calculating the rotation angle θ, retain the calculated rotation angle.

[0061] S134', when the x-coordinate of the top left vertex is P1 x x-coordinate P2 of the top right vertex x Half of the sum is greater than or equal to the center coordinate C of the human detection box. x When the rotation angle θ is corrected, the corrected rotation angle θ' is equal to 360 degrees minus the calculated rotation angle θ, so that the corrected rotation angle is between 0 and 2π.

[0062] S135, decompose the rotation angle of the human body detection frame into angle segments and angle segment offsets.

[0063] In this embodiment, S135 includes decomposing the rotation angle into angle segment c and angle segment offset d by converting the rotation angle from a radian value to an angle value.

[0064] In this embodiment, the integer value of the angle value divided by 10 is the angle segment c, and the units digit of the angle value is the angle segment offset d.

[0065] For example, if the calculated rotation angle is 2.9°, then angle = radians * 180 / pi, which is 2.9 * 180 / 3.14 ≈ 166. Converting to an angle value and rounding it down gives 166. Therefore, angle segment c is 16 and angle segment offset d is 6.

[0066] In this embodiment, the human detection angle encoding method reduces the amount of model output data. Models using this encoding method can reduce bandwidth consumption throughout the inference process, thus improving application performance. It is particularly suitable for application on front-end devices.

[0067] This embodiment also provides an image training method based on the human body detection angle encoding method, the image training method including:

[0068] Receive top-view training images, extract features from the top-view training images, and output feature maps at different scales.

[0069] Feature maps of different sizes are fused together to form spliced ​​feature maps of different scales;

[0070] The results of the spliced ​​feature maps of different sizes are predicted and then decoded to obtain the decoding results. The decoding results include target heatmaps at different scales, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box, and the angle segment offset.

[0071] Based on the human detection angle encoding method, the encoding results of the training images are obtained, and the loss is calculated for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales.

[0072] The image training method based on human detection angle encoding provided in this embodiment will be described in detail below with reference to the illustrations. Please refer to... Figure 5 The diagram shows a flowchart of an image training method based on the human body detection angle encoding method in one embodiment. Figure 5 As shown, the image training method specifically includes the following steps:

[0073] S41 receives training images based on the top viewpoint, extracts features from the training images based on the top viewpoint, and outputs feature maps at different scales.

[0074] In this embodiment, the features extracted from the top-view training images are the low-level features of the images, including color features, basic texture features, etc.

[0075] In this embodiment, a pre-stored Backbone structure is used to extract features from the training images. The pre-stored Backbone structure is as follows: Figure 6 As shown, Conv (kernel size - number of output channels) represents a convolutional layer, MaxPooling represents a max-pooling layer, and both have a stride of 2. CSP (number of output channels) labels the convolutional module, and its structure is as follows. Figure 7 As shown, M is the number of output channels. SPP is spatial pyramid pooling, and its structure is as follows: Figure 8 As shown. By means of Figure 6 The backbone structure shown extracts features from the training image and outputs feature maps at three scales: small target feature map, medium target feature map, and large target feature map.

[0076] S42 merges feature maps of different sizes to form spliced ​​feature maps of different scales.

[0077] In this embodiment, using, as Figure 9 The pre-stored feature fusion structure shown performs feature fusion on the feature maps of small, medium, and large targets.

[0078] like Figure 9 S42 includes:

[0079] First, convolution (CSP3-256) is performed on the large target feature map input-3 to obtain the large target feature map fusion output-3.

[0080] Next, the large target feature map input-3 is convolved and upsampled, then concatenated with the medium target feature map input-2. This concatenation is followed by another convolution (CSP3-128) and then a final convolution to obtain the inversely fused feature maps of the large and medium targets. Simultaneously, the large target feature map input-3 is convolved and upsampled, then upsampled and concatenated with the medium target feature map input-2, and then concatenated with the small target feature map input-1. This convolution yields features fused across three scales. This resulting feature map is then downsampled and fused with the medium target feature map. Finally, the three feature maps are output separately. For example, the three scales are S8 / S16 / S32.

[0081] S43, predict the results of the stitched feature maps of different sizes, and decode the prediction results to obtain the decoding results; the decoding results include target heatmaps at different scales, the size of the human detection box, target classification, the angle segment of the rotation angle of the human detection box and the angle segment offset.

[0082] In this embodiment, the prediction results include large-scale prediction results, mesoscale prediction results, and small-scale prediction results.

[0083] For example, if the input image has dimensions (H, W) and corresponding height and width, the output will be results at three scales: large-scale target prediction result R1, with dimensions (51, H / 32, W / 32); medium-scale target prediction result R2, with dimensions (51, H / 16, W / 16); and small-scale target prediction result R3, with dimensions (51, H / 8, W / 8).

[0084] In the large-scale prediction results, medium-scale prediction results, and small-scale prediction results, each pixel has 51 channels; among them, the first channel is the confidence score, the second and third channels are the width and height of the human detection box, the fourth and fifth channels are the target classification score (in this embodiment, the target classification includes human body and human head), the sixth to the forty-first channels are the angle segment score of the rotation angle of the human detection box, and the 42nd to the 51st channels are the angle segment offset score of the rotation angle.

[0085] S43 includes:

[0086] Normalization and non-maximum suppression calculations are performed on the first channel; maxima are filtered using a detection confidence threshold, and the coordinate indices of the remaining maxima are set as the coordinates of the predicted targets, thus forming a set of predicted targets.

[0087] For each coordinate in the predicted target set, obtain the values ​​corresponding to the second and third channels to predict the width and height of the human detection box;

[0088] For each coordinate in the set of predicted targets, obtain the score values ​​corresponding to the 4th and 5th channels, and set the index of the maximum score value as the predicted category of the target;

[0089] For each coordinate in the predicted target set, obtain the angle value corresponding to the 6th to 41st channels, and set the index of the maximum angle value as the predicted angle segment of the human body detection box rotation angle;

[0090] For each coordinate in the predicted target set, obtain the angle segment offset value corresponding to the 42nd to 51st channels. The index of the maximum angle segment offset value is set as the predicted angle segment offset of the human body detection box rotation angle.

[0091] S44. Based on the human body detection angle encoding method, obtain the encoding result of the training image, and calculate the loss for the target heat map, the size of the human body detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human body detection box at different scales.

[0092] In this embodiment, S44 includes:

[0093] S441, based on the coordinates of the target in the training image and the coordinates of the predicted target, calculate the loss of the target heatmap.

[0094] Specifically, the formula for calculating the loss of the target heatmap is: Where n represents the total number of pixels in the feature map, y i is the coordinates of a target point in the training image, and p is the coordinates of the predicted target.

[0095] S442, Calculate the human detection box size loss based on the size of the human detection box in the training image and the predicted width and height of the human detection box.

[0096] Specifically, the formula for calculating the size loss of the human body detection frame is as follows: Among them, w i and h i To predict the width and height of the human detection bounding box, w p and h p This represents the size of the human detection bounding box based on the training image.

[0097] S443 calculates the loss for the target category based on the category of the training image and the predicted category.

[0098] Specifically, the loss for the target category is calculated using cross-entropy loss, and the formula is as follows: Where K is the total number of categories, and its value is 2. c It is a D-one-hot encoding. When the training image category is human, y c If the value is 1, then the value is 0; otherwise, the value is 0.c The predicted value for the predicted category.

[0099] S444 calculates the loss for the angle segments based on the angle segments of the training images and the predicted angle segments.

[0100] Specifically, the predicted result for the angle segment is an array of length 36. The loss for the angle segment is calculated using focal loss, and the calculation formula is as follows: Among them, angle p For the predicted angle segment, target i To train the encoding of the angle segment of the image, with a size of 1x36, an example image of the target's encoded value is shown below. Figure 10 As shown. Where σ takes the value 6. i is an index, ranging from [0, 35]. The formula for calculating f(i) is: Where r is the remaining length of the angle segment, calculated using the formula: r = 36 - g(o); g(o) is the angle segment offset. o = 18 - angle t , where angle t The angle segment for rotating the human detection bounding box in the training image.

[0101] S445, based on the angle segment offset of the training image and the predicted angle segment offset, calculates the loss of the angle segment offset.

[0102] The cross-entropy loss is used to calculate the angle segment offset loss, and the calculation formula is as follows: Where K represents the total number of categories for angle segment offset, and its value is 10. c It is a D-one-hot encoding. When the angle segment offset of the training image is any number from 0 to 9, y c If the value is 1, then the value is 0; otherwise, the value is 0. c To predict the offset of the angle segment.

[0103] S446, calculate the total loss, i.e.

[0104] The image training method described in this embodiment is adapted to the above-mentioned human target encoding method. The model structure is lightweight and suitable for deployment on front-end devices.

[0105] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following... Figure 1 The human body detection angle encoding method and / or such as Figure 5 The image training method.

[0106] At any possible level of technical detail, this application can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0107] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0108] The computer-readable program described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards these instructions to a computer-readable storage medium in the respective computing / processing device. The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as "C" or similar programming languages. Computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of this application.

[0109] Example 2

[0110] This embodiment provides a human body detection angle encoding system, including:

[0111] The acquisition module is used to acquire human detection boxes based on the top view.

[0112] The sorting module is used to sort the vertices of the human body detection box according to a preset order;

[0113] The encoding module is used to encode the human detection box according to the sorted vertices of the human detection box after the human detection box has rotated, so as to obtain the encoding result; the encoding result includes the size of the human detection box, the angle segment decomposed based on the rotation angle of the human detection box, and the angle segment offset.

[0114] This embodiment also provides an image training system based on a human body detection angle encoding system, the image training system comprising:

[0115] The feature extraction module is used to receive top-view training images, extract features from the top-view training images, and output feature maps at different scales.

[0116] The feature fusion module is used to fuse feature maps of different sizes to form spliced ​​feature maps of different scales.

[0117] The prediction module is used to predict the results of the stitched feature maps of different sizes and decode the prediction results to obtain the decoding results. The decoding results include target heatmaps at different scales, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box, and the angle segment offset.

[0118] The loss calculation module is used to obtain the encoding results of the training images based on the human detection angle encoding method, and to calculate the loss for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales.

[0119] The following will describe in detail the human body detection angle encoding system and the image training system based on the human body detection angle encoding system provided in this embodiment, with reference to the illustrations. Please refer to... Figure 11A The diagram shows the principle structure of a human body detection angle encoding system in one embodiment. Figure 11A As shown, the human body detection angle encoding system 1 includes an acquisition module 11, a sorting module 12, and an encoding module 13.

[0120] The acquisition module 11 is used to acquire a human body detection box based on a top view.

[0121] The sorting module 12 is used to sort the vertices of the human body detection box in a preset order.

[0122] The preset order of the vertices of the human body detection frame is from the top left vertex P1 to the top right vertex P2, from the top right vertex P2 to the bottom left vertex P3, and from the bottom right vertex P3 to the bottom left vertex P4.

[0123] The encoding module 13 is used to encode the human detection box according to the sorted vertices of the human detection box after the human detection box has rotated, so as to obtain the encoding result; the encoding result includes the size of the human detection box, the angle segment decomposed based on the rotation angle of the human detection box, and the angle segment offset.

[0124] The encoding module 13 is used to calculate the width of the human detection box based on the distance between the coordinates of the top-left and top-right vertices, and to calculate the length of the human detection box based on the distance between the coordinates of the top-left and bottom-left vertices; to calculate the horizontal and vertical lengths of the top-left and bottom-left vertices based on their coordinates; to deduce the rotation angle of the human detection box based on the calculated horizontal and vertical lengths; when half of the sum of the horizontal coordinates of the top-left and top-right vertices is less than the center coordinates of the human detection box, the calculated rotation angle is retained. When half of the sum of the horizontal coordinates of the top-left and top-right vertices is greater than or equal to the center coordinates of the human detection box, the rotation angle is corrected; the corrected rotation angle is equal to 360 degrees minus the calculated rotation angle, so that the corrected rotation angle is between 0 and 2π; and the rotation angle of the human detection box is decomposed into angle segments and angle segment offsets.

[0125] In this embodiment, the encoding module 13 converts the rotation angle from radians to degrees to decompose the rotation angle into angle segment c and angle segment offset d. The integer value of the angle value divided by 10 is the angle segment c, and the units digit of the angle value is the angle segment offset d.

[0126] Please see Figure 11B The diagram shows the principle structure of an image training system based on a human body detection angle coding system in one embodiment. Figure 11B As shown, the image training system 2 based on the human body detection angle coding system includes: a feature extraction module 21, a feature fusion module 22, a prediction module 23, and a loss calculation module 24.

[0127] The feature extraction module 21 is used to receive training images based on the top view, extract features from the training images based on the top view, and output feature maps of different scales.

[0128] In this embodiment, the feature extraction module 21 uses a pre-stored Backbone structure to extract features from the training images. The pre-stored Backbone structure is as follows: Figure 6 As shown, Conv (kernel size - number of output channels) represents a convolutional layer, MaxPooling represents a max-pooling layer, and both have a stride of 2. CSP (number of output channels) labels the convolutional module, and its structure is as follows. Figure 7 As shown, M is the number of output channels. SPP is spatial pyramid pooling, and its structure is as follows: Figure 8 As shown. By means of Figure 6 The backbone structure shown extracts features from the training image and outputs feature maps at three scales: small target feature map, medium target feature map, and large target feature map.

[0129] The feature fusion module 22 is used to fuse feature maps of different sizes to form spliced ​​feature maps of different scales.

[0130] In this embodiment, the feature fusion module 22 utilizes, for example... Figure 9 The pre-stored feature fusion structure shown performs feature fusion on the feature maps of small, medium, and large targets.

[0131] The prediction module 23 is used to predict the results of the stitched feature maps of different sizes and decode the prediction results to obtain the decoding results; the decoding results include target heatmaps at different scales, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box and the angle segment offset.

[0132] In this embodiment, the prediction results include large-scale prediction results, mesoscale prediction results, and small-scale prediction results.

[0133] In the large-scale prediction results, medium-scale prediction results, and small-scale prediction results, each pixel has 51 channels; among them, the first channel is the confidence score, the second and third channels are the width and height of the human detection box, the fourth and fifth channels are the target classification score (in this embodiment, the target classification includes human body and human head), the sixth to the forty-first channels are the angle segment score of the rotation angle of the human detection box, and the 42nd to the 51st channels are the angle segment offset score of the rotation angle.

[0134] Specifically, the prediction module 23 performs normalization calculation and non-maximum suppression calculation on the first channel; filters the maxima using the detection confidence threshold, sets the coordinate index of the remaining maxima as the coordinates of the prediction target, and forms a set of prediction targets;

[0135] The prediction module 23 obtains the values ​​corresponding to the second and third channels for each coordinate in the prediction target set in order to predict the width and height of the human detection box;

[0136] For each coordinate in the set of predicted targets, the prediction module 23 obtains the score value corresponding to the 4th and 5th channels, and sets the index of the maximum score value as the prediction category of the target.

[0137] The prediction module 23 obtains the angle value corresponding to the 6th to 41st channels for each coordinate in the prediction target set, and sets the index of the maximum angle value as the predicted angle segment of the human body detection box rotation angle.

[0138] For each coordinate in the set of predicted targets, the prediction module 23 obtains the angle segment offset value corresponding to the 42nd to 51st channels, and the index of the maximum angle segment offset value is set as the predicted angle segment offset of the human body detection box rotation angle.

[0139] The loss calculation module 24 is used to obtain the encoding results of the training images based on the human detection angle encoding method, and to calculate the loss for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales.

[0140] In this embodiment, the loss calculation module 24 performs loss calculation on the target heatmap based on the coordinates of the target in the training image and the coordinates of the predicted target.

[0141] Specifically, the formula for calculating the loss of the target heatmap is: Where n represents the total number of pixels in the feature map, y i is the coordinates of a target point in the training image, and p is the coordinates of the predicted target.

[0142] The loss calculation module 24 calculates the human detection box size loss based on the size of the human detection box in the training image and the predicted width and height of the human detection box.

[0143] Specifically, the formula for calculating the size loss of the human body detection frame is as follows: Among them, w i and h i To predict the width and height of the human detection bounding box, w p and h p This represents the size of the human detection bounding box based on the training image.

[0144] The loss calculation module 24 performs loss calculation on the target category based on the category of the training image and the predicted category.

[0145] Specifically, the loss for the target category is calculated using cross-entropy loss, and the formula is as follows: Where K is the total number of categories, and its value is 2. c It is a D-one-hot encoding. When the training image category is human, y c If the value is 1, then the value is 0; otherwise, the value is 0. c This represents the predicted value for the predicted category.

[0146] The loss calculation module 24 performs loss calculation on the angle segments based on the angle segments of the training image and the predicted angle segments.

[0147] Specifically, the predicted result for the angle segment is an array of length 36. The loss for the angle segment is calculated using focal loss, and the calculation formula is as follows: Among them, angle p For the predicted angle segment, target i To train the encoding of the angle segment of the image, with a size of 1x36, an example image of the target's encoded value is shown below. Figure 10 As shown. Where σ takes the value 6. i is an index, ranging from [0, 35]. The formula for calculating f(i) is: Where r is the remaining length of the angle segment, calculated using the formula: r = 36 - g(o); g(o) is the angle segment offset. o = 18 - angle t , where angle t The angle segment for rotating the human detection bounding box in the training image.

[0148] The loss calculation module 24 calculates the loss based on the angle segment offset of the training image and the predicted angle segment offset.

[0149] The cross-entropy loss is used to calculate the angle segment offset loss, and the calculation formula is as follows: Where K represents the total number of categories for angle segment offset, and its value is 10. c It is a D-one-hot encoding. When the angle segment offset of the training image is any number from 0 to 9, y c If the value is 1, then the value is 0; otherwise, the value is 0. c To predict the offset of the angle segment.

[0150] The loss calculation module 24 calculates the total loss, i.e.

[0151] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls, entirely in hardware, or partially in software calls via processing element calls, with some modules implemented in hardware. For example, module x can be a separate processing element or integrated into a chip within the system. Additionally, module x can be stored as program code in the system's memory, invoked and executed by a processing element. The implementation of other modules is similar. These modules can be fully or partially integrated together or implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the processor element or through software instructions. These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), one or more Field Programmable Gate Arrays (FPGAs), etc. When a module is implemented through processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. These modules can be integrated together to form a System-on-a-Chip (SOC).

[0152] Example 3

[0153] This embodiment provides a terminal, which includes: a processor, a memory, a transceiver, a communication interface and / or a system bus; the memory and the communication interface are connected to the processor and the transceiver through the system bus and complete mutual communication; the memory is used to store computer programs; the communication interface is used to communicate with other devices; the processor and the transceiver are used to run the computer programs, so that the terminal executes each step of the human body detection angle encoding method described above and / or the image training method based on the human body detection angle encoding method.

[0154] The system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0155] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0156] The scope of protection of the human body detection angle encoding / image training method described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0157] This invention also provides a human body detection angle encoding / image training system, which can implement the human body detection angle encoding / image training method described in this invention. However, the implementation device of the human body detection angle encoding / image training method described in this invention includes, but is not limited to, the structure of the human body detection angle encoding / image training system listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of this invention are included within the protection scope of this invention.

[0158] In summary, the human body detection angle encoding / image training method, system, terminal, and medium described in this invention have the following beneficial effects:

[0159] First, the human body detection angle encoding method / system described in this invention reduces the amount of model output data. Models using this encoding method can reduce bandwidth consumption throughout the inference process, thus improving application performance. It is particularly suitable for application on front-end devices.

[0160] Secondly, the image training method / system based on human body detection angle encoding described in this invention is adapted to the aforementioned human target encoding method / system, and its model structure is lightweight, making it suitable for deployment on front-end devices. Therefore, this invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0161] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for encoding human body detection angles, characterized in that, include: Obtain the human detection bounding box based on the top view; The vertices of the human body detection frame are sorted in a preset order; After the human body detection box rotates, the human body detection box is encoded according to the sorted vertices of the human body detection box to obtain the encoding result; the encoding result includes the size of the human body detection box, the angle segment decomposed based on the rotation angle of the human body detection box, and the angle segment offset; the rotation angle is converted from radian value to angle value; wherein, the integer value of the angle value divided by 10 is the angle segment, and the unit digit value of the angle value is the angle segment offset; The preset order of the vertices of the human body detection frame is from the top left vertex to the top right vertex, from the top right vertex to the bottom left vertex, and from the bottom right vertex to the bottom left vertex. The steps for encoding the human detection boxes based on the sorted vertices to obtain the encoding results include: Calculate the width of the human detection box based on the distance between the coordinates of the top left and top right vertices, and calculate the length of the human detection box based on the distance between the coordinates of the top left and bottom left vertices. Based on the coordinates of the top left and bottom left vertices, calculate the horizontal length and vertical length of the top left and bottom left vertices, and deduce the rotation angle of the human body detection frame based on the calculated horizontal and vertical lengths. When half of the sum of the x-coordinates of the top left and top right vertices is less than the center coordinates of the human detection box, the calculated rotation angle is retained. When half of the sum of the x-coordinates of the top left and top right vertices is greater than or equal to the center coordinates of the human detection box, the rotation angle is corrected; the corrected rotation angle is equal to 360 degrees minus the calculated rotation angle. The rotation angle of the human body detection frame is decomposed into angle segments and angle segment offsets.

2. An image training method based on the human body detection angle encoding method of claim 1, characterized in that, The image training method includes: Receive top-view training images, extract features from the top-view training images, and output feature maps at different scales. Feature maps of different sizes are fused together to form spliced ​​feature maps of different scales; The results of the spliced ​​feature maps of different sizes are predicted and then decoded to obtain the decoding results. The decoding results include target heatmaps at different scales, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box, and the angle segment offset. Based on the human detection angle encoding method, the encoding results of the training images are obtained, and the loss is calculated for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales.

3. The image training method according to claim 2, characterized in that, The prediction results include large-scale prediction results, meso-scale prediction results, and small-scale prediction results. In the large-scale prediction results, medium-scale prediction results, and small-scale prediction results, each pixel has 51 channels; among them, the first channel is the confidence score, the second and third channels are the width and height of the human detection box, the fourth and fifth channels are the target classification score, the sixth to the forty-first channels are the angle segment score of the human detection box rotation angle, and the forty-second to the fifty-first channels are the angle segment offset score of the rotation angle.

4. The image training method according to claim 3, characterized in that, The steps for predicting the results of spliced ​​feature maps of different sizes and decoding the prediction results to obtain the decoding results include: Normalization and non-maximum suppression calculations are performed on the first channel; maxima are filtered using a detection confidence threshold, and the coordinate indices of the remaining maxima are set as the coordinates of the predicted targets, thus forming a set of predicted targets. For each coordinate in the predicted target set, obtain the values ​​corresponding to the second and third channels to predict the width and height of the human detection box; For each coordinate in the set of predicted targets, obtain the score values ​​corresponding to the 4th and 5th channels, and set the index of the maximum score value as the predicted category of the target; For each coordinate in the predicted target set, obtain the angle value corresponding to the 6th to 41st channels, and set the index of the maximum angle value as the predicted angle segment of the human body detection box rotation angle; For each coordinate in the predicted target set, obtain the angle segment offset value corresponding to the 42nd to 51st channels. The index of the maximum angle segment offset value is set as the predicted angle segment offset of the human body detection box rotation angle.

5. The image training method according to claim 4, characterized in that, The steps for obtaining the encoding results of the training images based on the human detection angle encoding method, and calculating the loss for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales, include: Based on the coordinates of the target in the training images and the coordinates of the predicted target, the loss of the target heatmap is calculated. Calculate the human detection box size loss based on the size of the human detection box in the training image and the width and height of the predicted human detection box. Based on the categories of the training images and the predicted categories, the loss is calculated for the target category; Based on the angle segments of the training images and the predicted angle segments, the loss is calculated for the angle segments. Based on the angle segment offset of the training image and the predicted angle segment offset, the loss of the angle segment offset is calculated.

6. A human body detection angle coding system, characterized in that, include: The acquisition module is used to acquire human detection boxes based on the top view. The sorting module is used to sort the vertices of the human body detection box according to a preset order; The encoding module is used to encode the human detection box based on the sorted vertices of the human detection box after it has rotated, in order to obtain an encoding result. The encoding result includes the size of the human detection box, the angle segments decomposed based on the rotation angle of the human detection box, and the angle segment offset. The rotation angle is converted from a radian value to an angle value. The integer value of the angle value divided by 10 is the angle segment, and the units digit of the angle value is the angle segment offset. The preset order of the vertices of the human detection box is from the top left vertex to the top right vertex, from the top right vertex to the bottom left vertex, and from the bottom right vertex to the bottom left vertex. The step of encoding the human detection box based on the sorted vertices to obtain the encoding result includes: calculating the distance between the coordinates of the top left vertex and the top right vertex. The width of the human detection frame is calculated, and its length is determined by the distance between the coordinates of the top-left and bottom-left vertices. The horizontal and vertical lengths of the top-left and bottom-left vertices are calculated based on their coordinates, and the rotation angle of the human detection frame is derived from these calculated lengths. If half the sum of the horizontal coordinates of the top-left and top-right vertices is less than the center coordinates of the human detection frame, the calculated rotation angle is retained. If half the sum of the horizontal coordinates of the top-left and top-right vertices is greater than or equal to the center coordinates of the human detection frame, the rotation angle is corrected. The corrected rotation angle is equal to 360 degrees minus the calculated rotation angle. The rotation angle of the human detection frame is decomposed into angle segments and angle segment offsets.

7. An image training system based on the human body detection angle encoding system of claim 6, characterized in that, The image training system includes: The feature extraction module is used to receive top-view training images, extract features from the top-view training images, and output feature maps at different scales. The feature fusion module is used to fuse feature maps of different sizes to form spliced ​​feature maps of different scales. The prediction module is used to predict the results of the stitched feature maps of different sizes and decode the prediction results to obtain the decoding results. The decoding results include target heatmaps at different scales, the size of the human detection box, the target classification, the angle segment of the rotation angle of the human detection box, and the angle segment offset. The loss calculation module is used to obtain the encoding results of the training images based on the human detection angle encoding method, and to calculate the loss for the target heatmap, the size of the human detection box, the target classification, the angle segment and the angle segment offset of the rotation angle of the human detection box at different scales.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the human body detection angle encoding method of claim 1 and / or the image training method of any one of claims 2 to 5.

9. A terminal, characterized in that, include: Processor and memory; The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the terminal to perform the human body detection angle encoding method as described in claim 1 and / or the image training method as described in any one of claims 2 to 5.

Images