A radar-based posture detection method and device, computer equipment and medium

Abstract

Embodiments of the present application provide a radar-based posture detection method and device, computer equipment and a medium. The method comprises: converting point cloud data obtained by a radar into a heat map and a grayscale map, then performing target detection on the heat map and the grayscale map respectively to obtain target detection images of the two maps, then performing action detection on the target detection images of the two maps through an action detection network model to obtain two detection results, and finally determining the posture of a target object according to the two detection results. This detection method converts point cloud data into heat maps and grayscale maps with small data volume and concentrated distribution, and combines a target detection neural network model for intelligent recognition, so that the radar can more accurately recognize the posture of a target object through point cloud data.

Description

Technical Field

[0001] This invention relates to the field of radar detection technology, and in particular to a radar-based attitude detection method, apparatus, computer equipment, and medium. Background Technology

[0002] Currently, people, such as the elderly, are usually monitored by installing surveillance cameras to detect potential dangers like falls. However, this method of fall detection can lead to privacy breaches, and objects such as blankets and curtains in the bedroom may obstruct the viewer's view, resulting in incorrect identification by the surveillance camera.

[0003] Therefore, technologies that use radar to replace surveillance cameras have emerged on the market. Radar can detect target objects through obstacles and has unique advantages in human posture detection. However, the point cloud data acquired by radar is large in volume and scattered in distribution, making it ineffective for identifying the posture of target objects through point cloud data. Summary of the Invention

[0004] This invention provides a radar-based attitude detection method, apparatus, computer equipment, and medium, aiming to solve the problem that existing radars have poor performance in identifying target attitudes during attitude detection.

[0005] In a first aspect, embodiments of the present invention provide a radar-based attitude detection method, comprising:

[0006] Radar is used to detect target objects within the detection area to obtain point cloud data;

[0007] The point cloud data is converted into heatmaps and grayscale images;

[0008] Target detection is performed on the heatmap and grayscale image respectively to obtain a first target detection image with target location corresponding to multiple frames of the heatmap and a second target detection image with target location corresponding to multiple frames of the grayscale image.

[0009] An action detection network model is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image to obtain a first detection result and a second detection result.

[0010] The pose of the target object is determined based on the first detection result and the second detection result.

[0011] Secondly, embodiments of the present invention provide a radar-based attitude detection device, comprising:

[0012] The object detection unit is used to detect target objects within the detection area using radar and obtain point cloud data;

[0013] A conversion unit is used to convert the point cloud data into a heat map and a grayscale image;

[0014] The target detection unit is used to perform target detection on the heat map and the grayscale image respectively, to obtain a first target detection image with target location corresponding to multiple frames of the heat map and a second target detection image with target location corresponding to multiple frames of the grayscale image.

[0015] An action detection unit is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image using an action detection network model, and obtain a first detection result and a second detection result.

[0016] A determining unit is used to determine the pose of the target object based on the first detection result and the second detection result.

[0017] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the radar-based attitude detection method as described in the first aspect.

[0018] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the radar-based attitude detection method as described in the first aspect.

[0019] This invention provides a radar-based attitude detection method, apparatus, computer device, and medium. This embodiment converts point cloud data acquired by radar into heatmaps and grayscale images, then performs target detection on both the heatmaps and grayscale images respectively, obtaining two target detection images. Next, an action detection network model is used to perform action detection on the two target detection images, obtaining two detection results. Finally, the attitude of the target object is determined based on the two detection results. This detection method, by converting point cloud data into smaller heatmaps and grayscale images and combining them with a target detection neural network model for intelligent recognition, enables radar to more accurately identify the attitude of target objects from point cloud data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A schematic flowchart of a radar-based attitude detection method provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of a sub-process of a radar-based attitude detection method provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of another sub-process of a radar-based attitude detection method provided in an embodiment of the present invention;

[0024] Figure 4 A schematic block diagram of a radar-based attitude detection device provided in an embodiment of the present invention;

[0025] Figure 5 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0029] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items, and all possible combinations, and includes such combinations.

[0030] Please see Figure 1 , Figure 1 This is a flowchart illustrating a radar-based attitude detection method according to an embodiment of the present invention. The method may include steps S101 to S105:

[0031] S101. Use radar to detect target objects within the detection area and obtain point cloud data;

[0032] In step S101, the radar is placed at the edge of the detection area. When a human body (i.e., the target object) enters the detection area, the radar detects the human body and obtains point cloud data. The point cloud data includes the coordinates of multiple three-dimensional points, Doppler velocity, and their signal-to-noise ratio (SNR). Points with an SNR lower than a preset value are then removed, achieving the effect of filtering out useless point cloud data. The radar is preferably a millimeter-wave radar because it has advantages such as good stability and low cost, and can detect target objects through certain obstacles, giving it a unique advantage in human posture detection.

[0033] Specifically, step S101 also includes:

[0034] At least two radars are used to detect target objects within the detection area to obtain point cloud data; wherein the at least two radars are located at different angles within the detection area.

[0035] In this embodiment, it is preferred to use at least two millimeter-wave radars to detect the target object. This embodiment uses two millimeter-wave radars for specific explanation. The two millimeter-wave radars are placed at the edge of the detection area, with an angle of about 90 degrees between them. Ideally, one millimeter-wave radar detects the front of the target object and the other detects the side of the target object. By acquiring point cloud data through the two millimeter-wave radars, the target object is detected at different angles within the detection area, making the detection results more accurate.

[0036] In some implementations, a person is defined to be moving in an enclosed area. One millimeter-wave radar is installed on the top wall of the enclosed area, and another millimeter-wave radar is installed on the side wall. The millimeter-wave radar installed on the top wall is mainly used to identify changes in the person's horizontal plane, while the millimeter-wave radar installed on the side wall is mainly used to identify changes in the person's vertical plane, so as to obtain more comprehensive posture information and improve the accuracy of posture recognition.

[0037] S102. Convert the point cloud data into a heat map and a grayscale image;

[0038] Specifically, step S102 includes the following steps:

[0039] A preset original heatmap with a predetermined color space and a predetermined resolution is provided, wherein the predetermined color space includes hue, saturation, and brightness;

[0040] Obtain a predetermined number of point cloud data frames, convert the x-axis data of the point cloud data into the x-axis data of the original heatmap, and convert the z-axis data of the point cloud data into the y-axis data of the original heatmap;

[0041] An intermediate point is set for the y-axis data of the point cloud data. The y-axis data that does not exceed the intermediate point is plotted with a hue value of the first saturation, and the y-axis data that exceeds the intermediate point is plotted with a hue value of the second saturation, to obtain the final heat map.

[0042] In this embodiment, point cloud data is converted into a heatmap. Specifically, this is done using a color space with hue, saturation, and value, and an original heatmap with a resolution of 640*480. The colors in the original heatmap are drawn using Hue values ​​as the first and second saturation values ​​in the color space. By collecting 10 frames of millimeter-wave radar data (i.e., 10 frames of point cloud data), the x-axis data of the point cloud data is converted into the x-axis data of the original heatmap, and the z-axis data is converted into the y-axis data. The y-axis data of the point cloud data is then plotted with a 2-meter midpoint. Point cloud data smaller than 2 meters are plotted with Hue values ​​as the first saturation value, and point cloud data larger than 2 meters are plotted with Hue values ​​as the second saturation value, resulting in the final heatmap. Saturation refers to the vividness of a color, also known as its purity. Therefore, in this embodiment, the first saturation can be set to 0 degrees, and the second saturation to 240 degrees. Point cloud data smaller than 2 meters will be plotted with a Hue value of 0 degrees, while point cloud data larger than 2 meters will be plotted with a Hue value of 240 degrees. This makes the point cloud data closer to the radar appear redder, and the point cloud data farther from the radar appear bluer. Thus, the heatmap better reflects the relevant features of the point cloud data, resulting in better recognition performance after input into the subsequent neural model. The preset resolution, preset frame rate, intermediate point settings, and the settings for the first and second saturations can all be set according to specific scenarios and requirements.

[0043] Specifically, step S102 further includes the following steps:

[0044] A pre-defined original grayscale image with a predetermined resolution;

[0045] The x-axis data of the predetermined number of point cloud data is converted into the x-axis data of the original grayscale image, the z-axis data of the point cloud data is converted into the y-axis data of the original grayscale image, and the y-axis data of the point cloud data is converted into grayscale values ​​to obtain the final grayscale image.

[0046] In this embodiment, the point cloud data is converted into a grayscale image. Specifically, this is done by using an original grayscale image with a preset resolution of 640*480, collecting 10 frames of data from a millimeter-wave radar (i.e., 10 frames of point cloud data), converting the x-axis data of the point cloud data to the x-axis data of the original grayscale image, converting the z-axis data of the point cloud data to the y-axis data of the original grayscale image, and converting the y-axis data of the point cloud data to the grayscale value of the original grayscale image to obtain the final grayscale image. The preset number of frames and the preset resolution can be set according to the specific scenario.

[0047] S103. Perform target detection on the heat map and grayscale image respectively to obtain a first target detection image with target location corresponding to multiple frames of the heat map and a second target detection image with target location corresponding to multiple frames of the grayscale image.

[0048] In this embodiment, a target detection neural network model needs to be established in advance, and target detection is performed on the heatmap and grayscale image respectively using this model. Figure 2 As shown, the specific steps of target detection include S201-S204:

[0049] S201. Extract feature maps from the input image using a convolutional layer, a first activation function, and a first pooling layer; wherein the input image is a heatmap or a grayscale image;

[0050] In this embodiment, feature extraction is performed on both the heatmap and the grayscale image through a convolutional layer, a first activation function, and a first pooling layer to obtain feature maps corresponding to the heatmap and grayscale images.

[0051] S202. Multiple feature anchors in the feature map are determined through the region proposal network layer. The attribute of each feature anchor in the feature map is determined through a binary classification activation function. The position of the feature anchor with a positive attribute is corrected to obtain the coordinates of the target detection box.

[0052] In this embodiment, a positive attribute for a feature anchor point indicates that it belongs to the target anchor point. If it belongs to the target anchor point, the position of the feature anchor point is corrected using a correction anchor point in order to obtain the coordinates of the target detection box.

[0053] S203. Using a region pooling layer, extract the target feature map from the feature map based on the coordinates of the target detection box;

[0054] S204. Calculate the target category of the target feature map through a fully connected layer. If the target category is a target object, determine the target location through the region proposal network layer.

[0055] In this embodiment, after the target feature map is fed into the fully connected layer, the probability of the target feature map belonging to each category is output. The highest probability is taken as the target category required in this embodiment. In this embodiment, there is only one category: target object (also called target location or target point cloud location). For example, after the target feature map passes through the fully connected layer, it outputs a probability value of 0.86. If the probability threshold is preset to 0.8, and the probability output by the fully connected layer is greater than the probability threshold of 0.8, then the target feature map belongs to the target category required in this embodiment. Then, the final target location is determined by the regional network proposal layer, thereby obtaining multiple frames of detection images with target locations (i.e., the first detection image and the second detection image).

[0056] S104. Using an action detection network model, action detection is performed on multiple frames of the first target detection image and multiple frames of the second target detection image to obtain a first detection result and a second detection result.

[0057] In this embodiment, an action detection network model is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image, such as... Figure 3 As shown, the specific steps include S301-S305:

[0058] S301. Convolve the first target detection image through the slow branch of the action detection network model to obtain the first spatial semantic information;

[0059] S302. Convolve the second target detection image through the slow branch of the action detection network model to obtain the second spatial semantic information;

[0060] S303. Convolve the first target detection image through the fast branch of the action detection network model to obtain the first motion information;

[0061] S304. Convolve the second target detection image through the fast branch of the action detection network model to obtain the second motion information;

[0062] S305. The first spatial semantic information and the first motion information are dimensionally transformed and fused through lateral connection to obtain a first detection result; the second spatial semantic information and the second motion information are dimensionally transformed and fused through lateral connection to obtain a second detection result.

[0063] This embodiment uses a fast and slow neural network model to perform action detection on the first and second target detection images. The slow and fast branches of the fast and slow neural network process and analyze the spatial and temporal dimensions, respectively. The fast and slow neural network algorithm as a whole consists of two convolutional branches: a slow branch and a fast branch.

[0064] The slow branch learns spatial semantic information, also known as the spatial location of the target category, using fewer frames and a larger number of channels. The slow branch uses a large step size τ to acquire video frames, typically set to 16. For a video with a frame rate of 30fps, the slow branch acquires approximately 2fps of video in about 1 second, meaning it performs 2 samples per second (T = 2). The slow branch has D channels.

[0065] The fast branch learns motion information, also known as the action being performed by the target category, using a larger number of frames and fewer channels. The fast branch uses a small stride to acquire video frames, τ / α, where α is typically set to 8. Therefore, for a video at 30fps, the fast branch can acquire 16fps of video in 1 second (i.e., αT). The fast branch has βD channels, which maintains lightweight design by using a smaller number of channels; β is typically set to 1 / 8.

[0066] In some implementations, the number of channels refers to the number of channels used to learn feature representations in the slow and fast branches. The number of channels determines the capacity and representational power of the network model.

[0067] Both the slow and fast branches use a 3D convolutional residual neural network model to convolve the first or second object detection image, respectively. At the end of each branch, global pooling is performed before feeding the data into a fully connected network layer for class prediction. The slow branch obtains the spatial location (i.e., spatial semantic information) used to predict the class to be detected, while the fast branch obtains the action being performed to predict the class to be detected (i.e., motion information). The spatial semantic information and motion information are then fused. However, because the feature dimensions of the two branches are inconsistent, the fast branch uses {αT, S}. 2 The slow branch is {T, S}, while the slow branch is {T, S}. 2 Therefore, fast and slow neural networks need to perform dimensionality transformation on spatial semantic information and motion information, changing the dimensional features of the fast branch from {αT, S}. 2 ,βC} is transformed into {T,S} by the reshape function. 2 The features of the fast branch are then fed into the slow branch through lateral connections for fusion (generally, feature addition is used, i.e., the feature vectors of the fast branch and the slow branch are added together) to obtain the fusion result. The fusion result is then input into the fully connected layer to obtain the confidence of each pose, and the pose with the highest confidence is taken as the detection result.

[0068] S105. Determine the pose of the target object based on the first detection result and the second detection result.

[0069] In some implementations, when the target object's posture is the target posture, the electronic device can be controlled to issue an alarm.

[0070] In some implementations, the target posture is falling.

[0071] For the same millimeter-wave radar, if the detection results corresponding to the heatmap and grayscale image (i.e., the first detection result and the second detection result) are the same, it is considered reliable, and the detection result is output. When the output result is a fall, an alarm is issued. If the first detection result and the second detection result are different (e.g., one is a fall and the other is not a fall), it is considered unreliable, and the detection continues. Alternatively, if both the first detection result and the second detection result are "not a fall," it is considered reliable, and the detection continues.

[0072] Therefore, assuming that only one millimeter-wave radar is used for detection, the way to determine the attitude is relatively simple. It is only necessary to determine whether the first detection result and the second detection result are both a fall. If so, an alarm can be issued.

[0073] If at least two millimeter-wave radars are used to detect the target object in the detection area, the judgment steps are different from those in the previous embodiment. Specifically, step S105 includes: determining whether the first detection result and the second detection result of one of the radars are both target attitudes. If so, the output result of the corresponding radar is set as the target attitude. If either the first detection result or the second detection result is not a target attitude, the output result of the corresponding radar is set as a non-target attitude.

[0074] Determine whether the first and second detection results of another radar are both the target attitude. If so, set the output result of the corresponding radar to the target attitude. If either the first or second detection result is not the target attitude, set the output result of the corresponding radar to the non-target attitude.

[0075] An alarm is issued when both radars output the target attitude.

[0076] This embodiment uses two millimeter-wave radars for specific illustration. The output result of either of these two individual millimeter-wave radars can refer to the judgment method for the same millimeter-wave radar mentioned earlier. That is, if the first detection result and the second detection result are the same, it is considered reliable and the detection result is output. When the output result is a fall, an alarm is issued. If the first detection result and the second detection result are different (for example, one is a fall and the other is not a fall), it is considered unreliable and the detection is returned to continue. Alternatively, if both the first detection result and the second detection result are not a fall, it is considered reliable and the detection is returned to continue.

[0077] If the outputs of the two millimeter-wave radars are identical, the outputs are considered reliable, and the final inference result is output. If the output indicates a fall, an alarm is issued; if the output indicates no fall (i.e., both millimeter-wave radars output "no fall"), the detection is repeated. If the outputs of the two millimeter-wave radars are different, for example, one outputs "fall" and the other outputs "no fall," the outputs are considered unreliable, and the detection process returns to continue.

[0078] This invention also provides a radar-based attitude detection device for performing any of the aforementioned embodiments of the radar-based attitude detection method. Specifically, please refer to... Figure 4 , Figure 4 This is a schematic block diagram of a radar-based attitude detection device provided in an embodiment of the present invention. The device 400 includes an object detection unit 410, a conversion unit 420, a target detection unit 430, a motion detection unit 440, and a determination unit 450.

[0079] The object detection unit 410 is used to detect target objects within the detection area using radar to obtain point cloud data;

[0080] The conversion unit 420 is used to convert the point cloud data into a heat map and a grayscale image;

[0081] The target detection unit 430 is used to perform target detection on the heat map and the grayscale image respectively, to obtain a first target detection image with target location corresponding to multiple frames of the heat map and a second target detection image with target location corresponding to multiple frames of the grayscale image.

[0082] The action detection unit 440 is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image using an action detection network model, and obtain a first detection result and a second detection result.

[0083] The determining unit 450 is used to determine the pose of the target object based on the first detection result and the second detection result.

[0084] In one embodiment, the object detection unit 410 includes:

[0085] A detection unit is used to detect target objects within a detection area using at least two radars to obtain point cloud data; wherein the at least two radars are located at different angles within the detection area.

[0086] In one embodiment, the conversion unit 420 includes:

[0087] The first preset unit is used to preset an original heatmap with a predetermined color space and a predetermined resolution, wherein the predetermined color space includes hue, saturation and brightness;

[0088] The first conversion unit is used to acquire point cloud data of a predetermined number of frames, convert the x-axis data of the point cloud data into the x-axis data of the original heatmap, and convert the z-axis data of the point cloud data into the y-axis data of the original heatmap.

[0089] The drawing unit is used to set an intermediate point for the y-axis data of the point cloud data, draw the y-axis data that does not exceed the intermediate point with a hue value of the first saturation, and draw the y-axis data that exceeds the intermediate point with a hue value of the second saturation, to obtain the final heat map.

[0090] In one embodiment, the conversion unit 420 further includes:

[0091] The second preset unit is used to preset an original grayscale image with a predetermined resolution;

[0092] The second conversion unit is used to convert the x-axis data of the predetermined number of point cloud data into the x-axis data of the original grayscale image, convert the z-axis data of the point cloud data into the y-axis data of the original grayscale image, and convert the y-axis data of the point cloud data into grayscale values ​​to obtain the final grayscale image.

[0093] In one embodiment, the target detection unit 430 includes:

[0094] The first extraction unit is used to extract feature maps of the input image using a convolutional layer, a first activation function, and a first pooling layer; wherein the input image is a heatmap or a grayscale image;

[0095] The acquisition unit is used to determine multiple feature anchors in the feature map through the region proposal network layer, determine the attribute of each feature anchor in the feature map through a binary classification activation function, correct the position of the feature anchor with positive attribute, and obtain the coordinates of the target detection box.

[0096] The second extraction unit is used to extract the target feature map from the feature map based on the coordinates of the target detection box using a region pooling layer;

[0097] The computation unit is used to calculate the target category of the target feature map through a fully connected layer. If the target category is a target object, the target location is then determined through the region proposal network layer.

[0098] In one embodiment, the motion detection unit 440 includes:

[0099] The first convolutional unit is used to convolve the first target detection image through the slow branch of the action detection network model to obtain the first spatial semantic information;

[0100] The second convolutional unit is used to convolve the second target detection image through the slow branch of the action detection network model to obtain the second spatial semantic information;

[0101] The third convolutional unit is used to convolve the first target detection image through the fast branch of the action detection network model to obtain the first motion information;

[0102] The fourth convolutional unit is used to convolve the second target detection image through the fast branch of the action detection network model to obtain the second motion information;

[0103] The dimension conversion unit is used to perform dimension conversion on the first spatial semantic information and the first motion information, and fuse them through lateral connection to obtain a first detection result, and to perform dimension conversion on the second spatial semantic information and the second motion information, and fuse them through lateral connection to obtain a second detection result.

[0104] In one embodiment, the determining unit 450 includes:

[0105] The first judgment unit is used to determine whether the first detection result and the second detection result of one of the radars are both target attitudes. If so, the output result of the corresponding radar is set to the target attitude. If either the first detection result or the second detection result is not the target attitude, the output result of the corresponding radar is set to a non-target attitude.

[0106] The second judgment unit is used to determine whether the first detection result and the second detection result of another radar are both target attitudes. If so, the output result of the corresponding radar is set to the target attitude. If either the first detection result or the second detection result is not the target attitude, the output result of the corresponding radar is set to a non-target attitude.

[0107] The alarm unit is used to issue an alarm when the output results of both radars show the target attitude.

[0108] like Figure 5 As shown, in another embodiment of the present invention, a computer device 500 is provided, including a memory 501, a processor 502, and a computer program 5031 stored in the memory 501 and executable on the processor 502. When the processor executes the computer program, it implements the aforementioned radar-based attitude detection method.

[0109] In another embodiment of the invention, a computer-readable storage medium 503 is provided. This computer-readable storage medium 503 may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program 5031, which, when executed by a processor 502, implements the aforementioned radar-based attitude detection method.

[0110] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0111] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0113] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0115] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A radar-based attitude detection method, characterized in that, include: Radar is used to detect target objects within the detection area to obtain point cloud data; The point cloud data is converted into heatmaps and grayscale images; Target detection is performed on the heatmap and grayscale image respectively to obtain a first target detection image with target location corresponding to multiple frames of the heatmap and a second target detection image with target location corresponding to multiple frames of the grayscale image. An action detection network model is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image to obtain a first detection result and a second detection result. The pose of the target object is determined based on the first detection result and the second detection result; The step of converting the point cloud data into a heatmap and a grayscale image includes: presetting an original heatmap with a predetermined color space and a predetermined resolution, wherein the predetermined color space includes hue, saturation, and brightness; acquiring a predetermined number of point cloud data frames; converting the x-axis data of the point cloud data into the x-axis data of the original heatmap; converting the z-axis data of the point cloud data into the y-axis data of the original heatmap; setting an intermediate point for the y-axis data of the point cloud data; drawing the y-axis data that does not exceed the intermediate point with a hue as the first saturation value; and drawing the y-axis data that exceeds the intermediate point with a hue as the second saturation value, thereby obtaining the final heatmap. The step of converting the point cloud data into a grayscale image and a grayscale image further includes: pre-setting an original grayscale image with a predetermined resolution; converting the x-axis data of a predetermined number of point cloud data into the x-axis data of the original grayscale image, converting the z-axis data of the point cloud data into the y-axis data of the original grayscale image, and converting the y-axis data of the point cloud data into grayscale values ​​to obtain the final grayscale image. The method of using radar to detect target objects within a detection area and obtain point cloud data includes using at least two radars to detect target objects within the detection area and obtain point cloud data; wherein the at least two radars are located at different angles within the detection area.

2. The radar-based attitude detection method according to claim 1, characterized in that, The step of performing target detection on the heatmap and grayscale image respectively includes: Feature maps of the input image are extracted using convolutional layers, a first activation function, and a first pooling layer; wherein the input image is a heatmap or a grayscale image. Multiple feature anchors in the feature map are determined by the region proposal network layer, the attribute of each feature anchor in the feature map is determined by the binary classification activation function, the position of the feature anchor with positive attribute is corrected, and the coordinates of the target detection box are obtained. The target feature map is extracted from the feature map based on the coordinates of the target detection box using a region pooling layer; The target feature map is processed by a fully connected layer to calculate the target category. If the target category is a target object, the target location is then determined by the region proposal network layer.

3. The radar-based attitude detection method according to claim 1, characterized in that, The step of using an action detection network model to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image includes: The first target detection image is convolved by the slow branch of the action detection network model to obtain the first spatial semantic information; The second target detection image is convolved by the slow branch of the action detection network model to obtain the second spatial semantic information; The first target detection image is convolved by the fast branch of the action detection network model to obtain the first motion information; The second target detection image is convolved by the fast branch of the action detection network model to obtain the second motion information; The first spatial semantic information and the first motion information are dimensionally transformed and fused through lateral connection to obtain a first detection result. The second spatial semantic information and the second motion information are dimensionally transformed and fused through lateral connection to obtain a second detection result.

4. The radar-based attitude detection method according to claim 1, characterized in that, Determining the pose of the target object based on the first detection result and the second detection result includes: Determine whether the first detection result and the second detection result of one of the radars are both the target attitude. If so, set the output result of the corresponding radar to the target attitude. If either the first detection result or the second detection result is not the target attitude, set the output result of the corresponding radar to the non-target attitude. Determine whether the first and second detection results of another radar are both the target attitude. If so, set the output result of the corresponding radar to the target attitude. If either the first or second detection result is not the target attitude, set the output result of the corresponding radar to the non-target attitude. An alarm is issued when both radars output the target attitude.

5. A radar-based attitude detection device, characterized in that, include: The object detection unit is used to detect target objects within the detection area using radar and obtain point cloud data; A conversion unit is used to convert the point cloud data into a heat map and a grayscale image; The target detection unit is used to perform target detection on the heat map and the grayscale image respectively, to obtain a first target detection image with target location corresponding to multiple frames of the heat map and a second target detection image with target location corresponding to multiple frames of the grayscale image. An action detection unit is used to perform action detection on multiple frames of the first target detection image and multiple frames of the second target detection image using an action detection network model, and obtain a first detection result and a second detection result. A determining unit is configured to determine the pose of the target object based on the first detection result and the second detection result; The conversion unit is specifically used to preset an original heatmap with a predetermined color space and a predetermined resolution, wherein the predetermined color space includes hue, saturation, and brightness; acquire a predetermined number of point cloud data, convert the x-axis data of the point cloud data into the x-axis data of the original heatmap, convert the z-axis data of the point cloud data into the y-axis data of the original heatmap; set an intermediate point for the y-axis data of the point cloud data, draw the y-axis data that does not exceed the intermediate point with a hue as the first saturation value, and draw the y-axis data that exceeds the intermediate point with a hue as the second saturation value, to obtain the final heatmap; The conversion unit is further specifically used to pre-set an original grayscale image with a predetermined resolution; convert the x-axis data of a predetermined number of point cloud data into the x-axis data of the original grayscale image; convert the z-axis data of the point cloud data into the y-axis data of the original grayscale image; and convert the y-axis data of the point cloud data into grayscale values ​​to obtain the final grayscale image. The object detection unit is specifically used to detect target objects within the detection area using at least two radars to obtain point cloud data; wherein the at least two radars are located at different angles within the detection area.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the radar-based attitude detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the radar-based attitude detection method as described in any one of claims 1 to 4.

Images