Method and device for processing radar point cloud for moving target detection

Abstract

To provide a method and a device for processing a radar point cloud for moving target detection.SOLUTION: The method includes: a step for performing point cloud preprocessing on radar point cloud of a plurality of frames, one radar frame at a time; a step for performing frame merging processing on a plurality of radar frames having undergone the point cloud preprocessing; and a step for performing point cloud postprocessing on a radar point cloud in the plurality of radar frames having undergone the frame merging processing. According to an embodiment of the present invention, it is possible to realize detection of a moving target and improve detection accuracy without the need for limiting operation categories.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present invention relates to the technical field of radar detection. [Background technology]

[0002] The method for detecting human body keypoints based on radar point clouds has been widely applied because it does not violate privacy, can operate in dark or partially occluded scenes, and can obtain the real spatial positions of human body keypoints.

[0003] Figure 1 is a schematic diagram of the application of the human body keypoint detection method based on radar point cloud. As shown in Figure 1, when the human body is moving, the corresponding point cloud can be obtained from the radar, and the human body keypoint information can be detected from the point cloud, which can be used for motion detection, behavior analysis, fall detection, etc.

[0004] In motion detection and behavior analysis, the motion and behavior can be determined by analyzing the relative positions and statistical information of key points. In fall detection, it can be determined whether the target has fallen based on the height information of the key point positions of the human body.

[0005] It should be noted that the above description of the technical background is for the purpose of making the technical solutions of the present invention clear and complete, and is described for the purpose of making those skilled in the art understand. These technical solutions are merely described as part of the background technology of the present invention, and are not well known to those skilled in the art. Summary of the Invention [Problem to be solved by the invention]

[0006] According to the discovery of the present inventors, in some situations, the radar can only obtain a part of the reflectance information of the human body at one time, and there exist differences in the distribution of point clouds in space and time for different movements. In addition, the quality of the radar point cloud input into the keypoint detection model directly affects the detection accuracy of the keypoints.

[0007] In view of at least one of the above technical problems or similar problems, embodiments of the present invention provide a method and apparatus for processing radar point clouds for moving target detection, which can improve detection accuracy. [Means for solving the problem]

[0008] In one aspect of an embodiment of the present invention, a radar point cloud processing device for detecting moving targets is provided, the device including: a pre-processing unit that performs point cloud pre-processing on radar point clouds of multiple frames, with the radar frame as a unit; a merging processing unit that performs frame merging processing on the multiple radar frames that have been subjected to point cloud pre-processing; and a post-processing unit that performs point cloud post-processing on the radar point clouds in the multiple radar frames that have been subjected to frame merging processing.

[0009] In another aspect of an embodiment of the present invention, there is provided a moving target detection device, the device including: the device described in the above embodiment that performs point cloud pre-processing on radar point clouds of multiple frames to obtain a point cloud of a total number of points to be merged; and a detection device that performs moving target detection on the point cloud of the total number of points to be merged using a moving target detection model to obtain the moving target.

[0010] In another aspect of an embodiment of the present invention, an electronic device includes a memory having a computer program stored therein, and a processor, wherein the processor is configured to execute the computer program to realize a method including the steps of: performing point cloud pre-processing on radar point clouds of multiple frames, with each frame being a unit of radar frame; performing frame merging processing on the multiple radar frames on which point cloud pre-processing has been performed; performing point cloud post-processing on the radar point clouds in the multiple radar frames on which frame merging processing has been performed, to obtain a point cloud of a total number of points to be merged; and performing moving target detection on the point clouds of the total number of points to be merged using a moving target detection model, to obtain moving targets.

[0011] One of the advantageous effects of the embodiment of the present invention is as follows: by taking a radar frame as a unit, performing point cloud pre-processing on the radar point clouds of multiple frames respectively, determining the radar frames to be point cloud merged according to a point cloud merging policy, and selecting the radar point clouds in the determined radar frames to be point cloud merged according to a predetermined policy as inputs of a moving target detection model, it is possible to realize the detection of moving targets and improve the detection accuracy without the need to limit the category of motion.

[0012] Certain embodiments of the invention are disclosed in detail, as set forth in the following description and in the drawings, which show how the principles of the invention may be employed, but are not intended to be limited in scope, and include all changes, modifications, and equivalents within the spirit and content of the appended claims. [Brief description of the drawings]

[0013] The drawings included herein are for understanding the embodiments of the present invention, constitute a part of this specification, and are for illustrating the embodiments of the present invention, and together with the description of the text, explain the principles of the present invention. Note that the drawings described herein are merely for illustrating the embodiments of the present invention, and those skilled in the art can easily obtain other drawings based on these drawings. [Figure 1] 1 is a schematic diagram of the application of a human body keypoint detection method based on radar point cloud; [Diagram 2] FIG. 2 is a schematic diagram of a set of radar points Pt for one frame. [Diagram 3] FIG. 2 is a schematic diagram of a continuous radar point cloud of N frames. [Figure 4] FIG. 2 is a schematic diagram of human body key points. [Diagram 5] FIG. 1 is a schematic diagram of conventional point cloud pre-processing. [Figure 6] 13 is a schematic diagram of contrasting point cloud merging results for a fixed number of frames. [Figure 7]FIG. 2 is a schematic diagram of an example method for processing radar point clouds for moving target detection according to an embodiment of the present invention. [Figure 8] FIG. 2 is a schematic diagram of an example of a method for processing radar point clouds according to an embodiment of the present invention. [Figure 9] FIG. 2 is a schematic diagram of an example of a method for processing radar point clouds according to an embodiment of the present invention. [Figure 10] FIG. 4 is a schematic diagram of another example of a radar point cloud processing method according to an embodiment of the present invention. [Figure 11] 2 is a schematic diagram of an example of a method for detecting a moving target according to an embodiment of the present invention; [Figure 12] FIG. 2 is a schematic diagram of an example of a keypoint detection model. [Figure 13] FIG. 2 is a schematic diagram of an example of a post-processing model. [Figure 14] FIG. 2 is a schematic diagram of an example of a radar point cloud processing device according to an embodiment of the present invention. [Figure 15] 1 is a schematic diagram of an example of a moving target detection device according to an embodiment of the present invention; [Figure 16] 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The above and other features of the present invention will become apparent from the drawings and the following description. The specification and drawings disclose specific embodiments of the present invention, i.e., some embodiments according to the principles of the present invention. It is to be understood that the present invention is not limited to the described embodiments, and the present invention includes all modifications, variations, and equivalents within the scope of the appended claims.

[0015] In the embodiments of the present invention, the terms "first" and "second" are used to distinguish different elements by name, and do not imply any spatial arrangement or temporal order of these elements, and the elements are not limited to these terms. The term "and / or" includes any one or more of the listed terms and combinations thereof. The terms "including," "including," and "having" refer to the presence of stated features, elements, components or members, but do not exclude the presence or addition of one or more other features, elements, components or members.

[0016] In the present embodiment, the singular forms "a", "the", "the" and the like include the plural, meaning "a kind" or "a class", and are not limited to "one". Additionally, the term "said" includes both the singular and the plural, unless the context clearly dictates otherwise. Additionally, the term "according to" means "at least in part depending on" and the term "based on" means "at least in part based on", unless the context clearly dictates otherwise.

[0017] Features described and / or shown with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, may be combined with features in the other embodiments, or may be used in place of features of the other embodiments. The terms "including" or "having" refer to the presence of a described feature, element, component or member, but do not exclude the presence or addition of one or more other features, elements, components or members.

[0018] In the embodiment of the present invention, the radar may be, but is not limited to, a millimeter wave (mmWave) radar. The radar emits electromagnetic waves through a transmitting antenna, and receives corresponding reflected waves (which may be referred to as radar echo information) after being reflected by different objects. By analyzing the radar echo information, information such as the position, radial moving speed, etc. of the object from the radar can be effectively extracted, which information can meet the needs of many application scenarios.

[0019] In the embodiment of the present invention, the detection target may be people of various ages, for example, the elderly, children, the elderly and / or nursing staff, children and / or guardians. The present invention is not limited thereto, and the detection target may be an animal with vital signs, a robot without vital signs, etc. The following will be described taking the human body as an example.

[0020] In an embodiment of the present invention, a single radar point cloud is represented as p=(s,v,I,x,y,z), where s is the frame sequence number, v is the Doppler velocity relative to the radar, I is the signal strength, and x, y, z are spatial coordinates.

[0021] In an embodiment of the present invention, the set of radar points in one frame is P={p j , 1≦j≦n}, where n is the number of radar point clouds in the current frame. Figure 2 shows the radar point cloud set P t FIG.

[0022] In an embodiment of the present invention, the N frames of consecutive radar point clouds are: (outside 1) TIFF2025014118000002.tif22170, where N is the number of radar point clouds in the current frame, and i is the frame sequence number representing time. The frame sequence number is in chronological order, and the larger it is, the later the corresponding point cloud data appears. Figure 3 is a schematic diagram of consecutive radar point clouds of N frames.

[0023] In the embodiment of the present invention, the key points of the human body refer to the spatial positions of the joints or organs of the human body, and H={(x k ,y k ,z k ), 1≦k≦m}, where (x k ,y k ,z k ) represents the spatial location of the kth keypoint, and m is the number of keypoints. Figure 4 is a schematic diagram of human body keypoints.

[0024] In conventional human body keypoint detection methods based on radar point clouds, the quality of the input radar point cloud directly affects the keypoint detection accuracy, and conventional operations process the radar point cloud in a trial and error manner and do not have a unified optimization policy.

[0025] Fig. 5 is a schematic diagram of conventional point cloud pre-processing. As shown in Fig. 5, the conventional point cloud pre-processing method selects a fixed number of frames, i.e., radar frames within a fixed period, and selects a fixed number of radar points per frame. In addition, for different targets and / or different actions, a unified pre-processing method is used, and the key point information cannot be accurately described.

[0026] Figure 6 is a schematic diagram of the comparison of point cloud merging results with a fixed number of frames. As shown in the right diagram of Figure 6(a), in the conventional point cloud pre-processing method, when the motion is fast, redundant interference information (N=10) is merged. As shown in the left diagram of Figure 6(b), when the motion is slow, information loss (N=5) occurs.

[0027] In view of the above problems, the present invention has been proposed. Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0028] <Example 1> An embodiment of the present invention provides a method for processing radar point clouds for moving target detection. Figure 7 is a schematic diagram of an example of a method for processing radar point clouds for moving target detection according to an embodiment of the present invention. As shown in Figure 7, the method includes the following steps:

[0029] Step 701: Taking a radar frame as a unit, point cloud pre-processing is performed on radar point clouds of multiple frames.

[0030] Step 702: Frame merging processing is performed on the multiple radar frames that have been subjected to point cloud pre-processing.

[0031] Step 703: Perform point cloud post-processing on the radar point clouds in the multiple radar frames that have been subjected to the frame merging process.

[0032] Note that the above FIG. 7 merely illustrates an embodiment of the present invention, and the present invention is not limited thereto. For example, the execution order between each step may be adjusted as appropriate, other steps may be added, or some steps may be deleted. Those skilled in the art may make appropriate modifications based on the above content, and are not limited to the description of the above FIG. 7.

[0033] According to this embodiment, point cloud pre-processing is performed on the radar point clouds of multiple frames, with the radar frame as a unit, the radar frames to be point cloud-merged are determined in accordance with a point cloud merging policy, and the radar point clouds in the determined radar frames to be point cloud-merged are selected as input to a moving target detection model in accordance with a predetermined policy, thereby making it possible to detect moving targets and improve detection accuracy without the need to limit the category of motion.

[0034] 8 is a schematic diagram of an example of a radar point cloud processing method according to an embodiment of the present invention. As shown in FIG. 8, in the embodiment of the present invention, pre-processing is performed on different radar frames respectively, for example, single-frame pre-processing-0 is performed on the i-th frame, single-frame pre-processing-1 is performed on the i-1-th frame, single-frame pre-processing-N is performed on the iN-1-th frame, and so on.

[0035] In the above embodiment, the operations of point cloud pre-processing performed on different radar frames may be the same or different, for example, the above single frame pre-processing-0, single frame pre-processing-1, ..., single frame pre-processing-N may be the same, different, or partially the same, so that there is no need to limit the categories of operations in the process of performing point cloud pre-processing, and the flexibility is higher.

[0036] In the above embodiment, the radar frames may be subjected to point cloud pre-processing, followed by point cloud frame merging processing, determining the radar frames to be merged, and then performing merged point cloud post-processing, i.e., selecting the radar point cloud in the determined merged radar frame as the input of a moving object detection model (e.g., a keypoint detection model).

[0037] In some embodiments, performing point cloud pre-processing on the radar point cloud of a single frame in step 701 may include the following steps.

[0038] Step S11: Filter the point cloud in the current radar frame.

[0039] Step S12: Determine the points to be reserved in the radar frame according to a predetermined policy.

[0040] N frames of continuous radar point cloud collection (outside 2) Take TIFF2025014118000003.tif22170 as an example, the radar point cloud for a single frame is P t ={p j , 1≦j≦n}, where n is the number of point clouds in a single frame and t is the frame sequence number.

[0041] In step S11, the point cloud may be filtered as follows.

[0042] P t '=filter(P t ) In the above embodiment, the point cloud may be filtered using any filtering method such as a spatial filtering method, a velocity filtering method, or a cluster filtering method. Here, the point cloud may be filtered using the spatial filtering method, and for example, a point cloud outside a predetermined range may be removed by filtering. The point cloud may be filtered using the velocity filtering method, and for example, a point cloud with a speed that is too fast and / or a point cloud with a speed that is too slow may be removed by filtering. The point cloud may be filtered using the cluster filtering method, and for example, a point cloud that has not become a cluster after clustering, or a point cloud with too few points that have become clusters by clustering may be removed by filtering. The specific filtering method to be used may be determined as necessary, and the present invention is not limited to these. In addition, for specific embodiments of the above spatial filtering method, velocity filtering method, cluster filtering method, etc., reference may be made to related art, and the description thereof will be omitted here.

[0043] In step S12, points in the radar frame to be retained may be determined according to a predetermined policy.

[0044] In some embodiments, the predetermined policy is that the number of points in a radar frame does not exceed a first threshold.

[0045] In the above embodiment, if the number of points in the filtered point cloud does not exceed the first threshold, the filtered point cloud is set as the point cloud to be reserved in the radar frame. If the number of points in the filtered point cloud exceeds the first threshold, the filtered point cloud is sampled, for example, farthest point sampling or downsampling is performed, to obtain the point cloud to be reserved in the radar frame. Here, the number of points in the radar frame to be reserved does not exceed the first threshold.

[0046] In some other embodiments, the predetermined policy is that the number of points in a radar frame does not exceed a predetermined percentage of the total number of points to be merged.

[0047] In the above embodiment, if the number of points after filtering does not exceed a predetermined ratio of the total number of points to be merged, the points after filtering are set as points to be reserved in the radar frame. If the number of points after filtering exceeds a predetermined ratio of the total number of points to be merged, the points after filtering are sampled, for example, by farthest point sampling or downsampling, to obtain points to be reserved in the radar frame. Here, the number of points to be reserved in the radar frame does not exceed a predetermined ratio of the total number of points to be merged.

[0048] In an embodiment of step S12, the predefined policies (e.g., first thresholds or predefined percentages) corresponding to different time-ordered radar frames may be the same or different, e.g., the first threshold / predefined percentage adopted for the radar point cloud in the (i-1)th frame is different from the first threshold / predefined percentage adopted for the radar point cloud in the i-th frame.

[0049] In the above embodiment, a predetermined policy (e.g., a first threshold or a predetermined ratio) is determined according to the time order of the radar point clouds of a single frame, and the point cloud P to be reserved in the frame is determined according to the predetermined policy. t '' may be determined and expressed as follows:

[0050] P t ''=select(P t ,t,M0) Here, M0 is the total number of points to be finally merged.

[0051] For example, the predetermined policy is that the number of radar point clouds in the current frame does not exceed m1. If it exceeds m1, m1 points are obtained by farthest point sampling. Also, for example, the predetermined policy is that the number of radar point clouds in the current frame does not exceed q(t)% of the total number of points to be merged. If it exceeds q(t)%, a number of points equal to or less than q(t)% of the total number of points to be merged is obtained by farthest point sampling.

[0052] In some embodiments, performing frame merging processing on the point cloud pre-processed radar frames in step 702 may include the following steps.

[0053] Step S21: A validity check is performed on the data of the current radar frame.

[0054] Step S22: If the data of the radar frame is valid, start from the radar frame and sequentially merge the point clouds in each previous radar frame within a certain number of frames until the number of point clouds is at least equal to the total number of points to be merged.

[0055] In step S21, it is determined whether to merge the radar data of the frame. For example, if the total number of point clouds of the current radar frame and a predetermined number of radar frames immediately before the current radar frame is less than a threshold (referred to as a second threshold), it is determined that the data of the current radar frame is invalid, and the data of the current radar frame is not merged; otherwise, it is determined that the data of the current radar frame is valid, and the data of the current radar frame is merged.

[0056] For example, the latest N1 frame (current radar frame P N If the total number of points in the radar frame (including the radar frame and the N1-1 radar frames before the radar frame) is less than M1 (a second threshold), determine that the data of the current radar frame is invalid, and do not merge the radar data of the frame. This is expressed by the following formula:

[0057]

number

[0058] Taking N1=3 and M1=30 as an example, if the sum of the number of point clouds in the latest three frames (including the current radar frame and the two radar frames before the current radar frame) is less than 30, it is determined that the data of the current radar frame is invalid, and the frame of the current radar frame is not merged.

[0059] In step S22, starting from the latest radar frame (current radar frame) among the valid radar frames, the point clouds in each radar frame immediately preceding the latest one are sequentially merged within a fixed number (constant number) of frames. For example, when radar point cloud data of N frames are merged in ascending order of frame number, and the total number of merged point clouds is equal to or greater than the total number of points to be merged M (M≧M0), the merging is stopped, the number of merged radar frames tn is obtained, and the point cloud set of this tn frame is calculated. (Outside 3) We get TIFF2025014118000005.tif20170, where tn ≤ N.

[0060] In the above embodiment, N frames is set as the maximum limit for frame merging, which allows the frames to be merged to be limited to a certain period, and prevents the detection result from being influenced by too long history information.

[0061] Although the above is an exemplary description of frame merging, the present invention is not limited thereto, and frame merging may be performed using other policies. For example, a certain number of frames immediately preceding the current radar frame may be directly merged without determining the validity of the data, and the total number of points to be merged may be obtained as a convergence condition.

[0062] In the above embodiment, in step 703, the total number of point clouds to be merged may be selected based on the spatial distribution characteristics and / or velocity distribution characteristics of the point clouds in the multiple radar frames after the merging process.

[0063] For example, all point clouds in multiple radar frames after merging processing are sampled, and, for example, farthest point sampling is performed to obtain a point cloud of the total number of points to be merged.

[0064] In addition, for example, the point clouds of a frame other than the last frame in time (which may be the current radar frame) among the multiple radar frames after the merging process are sampled, for example, by performing farthest point sampling, and the number of point clouds in that frame (referred to as MP) is determined, and then the point clouds in the remaining frames and the point cloud in that frame are merged to obtain a point cloud of the total number of points to be merged.

[0065] Still taking the above as an example, in the above embodiment, in the determined tn frame, M0 points may be selected and sampled based on the characteristics of the point cloud, such as spatial distribution, speed distribution, etc. For example, in all radar point clouds in the tn frame, M0 points are obtained by farthest point sampling. Alternatively, only the last frame (latest frame, e.g., current frame) is sampled, the number of point clouds in this frame is MP, and the remaining M0-MP point clouds in the remaining tn-1 frames are sequentially merged to obtain M0 points.

[0066] This is just one example; other post-processing or sampling methods may be used to process the radar point cloud of the tn frames to obtain as many points as required.

[0067] The following describes a radar point cloud processing method according to an embodiment of the present invention with reference to two examples.

[0068] 9 is a schematic diagram of an example of a radar point cloud processing method according to an embodiment of the present invention. In the example of FIG. 9, a spatial filtering method is used to filter a single frame point cloud, where m1=50, N=10, and M=M0=200. As shown in FIG. 9, the method includes the following steps:

[0069] Step 901: Perform single-frame pre-processing, for example, perform spatial filtering on the point cloud of a single frame, filter out radar points with heights of 2.3 m or more, and the number of radar points in the current radar frame should not exceed 50.

[0070] Step 902: Perform point cloud frame merging process, for example, merging within 10 frames, stopping when the total number of points is greater than 200, and obtain the number of frames tn in which point cloud merging has been performed.

[0071] Step 903: Perform merged point cloud post-processing, for example, retain all point clouds of previous tn-1 frames, and perform farthest point sampling for the last frame, so that the total number of the final point cloud is 200.

[0072] The resulting point cloud may then be input into a moving target detection model (e.g., a keypoint detection model) for post-processing.

[0073] 10 is a schematic diagram of another example of a radar point cloud processing method according to an embodiment of the present invention. In the example of FIG. 10, a spatial filtering method is used to filter a single frame point cloud, where N=10, M=300, and M0=200. As shown in FIG. 10, the method includes the following steps:

[0074] Step 1001: Perform single-frame pre-processing, for example, perform spatial filtering on the point cloud of a single frame, and filter out radar points with heights greater than or equal to 2.3 m.

[0075] Step 1002: Perform point cloud frame merging processing, for example, merging within 10 frames, stopping when the total number of points is greater than 300, and obtain the number of frames tn in which point cloud merging has been performed.

[0076] Step 1003: Perform merged point cloud post-processing, for example using farthest point sampling, to select 200 points.

[0077] The resulting point cloud may then be input into a moving target detection model (e.g., a keypoint detection model) for post-processing.

[0078] The above-mentioned embodiments are merely illustrative of the embodiments of the present invention, and the present invention is not limited thereto, and appropriate modifications may be made based on the above-mentioned various embodiments. For example, the above-mentioned embodiments may be used alone, or one or more of the above-mentioned embodiments may be used in combination.

[0079] According to this embodiment, point cloud pre-processing is performed on the radar point clouds of multiple frames, with the radar frame as a unit, the radar frames to be point cloud-merged are determined in accordance with a point cloud merging policy, and the radar point clouds in the determined radar frames to be point cloud-merged are selected as input to a moving target detection model in accordance with a predetermined policy, thereby making it possible to detect moving targets and improve detection accuracy without the need to limit the category of motion.

[0080] <Example 2> An embodiment of the present invention provides a method for detecting a moving target. Figure 11 is a schematic diagram of an example of a method for detecting a moving target according to an embodiment of the present invention. As shown in Figure 11, the method includes the following steps:

[0081] Step 1101: Perform point cloud pre-processing on radar point clouds of multiple frames to obtain a point cloud of the total number of points to be merged.

[0082] Step 1102: In the moving target detection model, moving target detection is performed on the point cloud of the total number of points to be merged, and the detection result of the moving target is obtained.

[0083] In the above embodiment, step 1101 may be realized by the method according to embodiment 1, which has already been described in embodiment 1, and the contents thereof are incorporated and the description thereof is omitted here.

[0084] In this embodiment, before entering into the detection model, the radar point cloud is first preprocessed by the method according to the first embodiment, thereby making it possible to detect moving targets without limiting the categories of movements and improve the detection accuracy.

[0085] In some embodiments, the moving target detection model is a keypoint detection model. In the above embodiments, the keypoint detection model may be used to perform keypoint detection on the point cloud of the total number of points to be merged to obtain keypoints of the human body.

[0086] 12 is a schematic diagram of an example of a keypoint detection model. As shown in FIG. 12, when an input point cloud is input into the keypoint detection model, the keypoint detection model performs keypoint detection on the input point cloud to obtain keypoints of the human body.

[0087] In the above embodiment, a keypoint detection model composed of modules such as conv1d, EdgeConv, Adaptive graph convolution, spatial attention+channel attention, and fully connected is used as an example, but the present invention is not limited to this, and the keypoint detection model may further include other structures (layers), or some structures (layers) may be deleted. For details, refer to related technologies.

[0088] In the above embodiment, taking predicting the three-dimensional coordinates (x, y, z) of 17 keypoints of the human body as an example, the keypoint detection model may be realized with other parameters according to the number of keypoints to be predicted.

[0089] In some embodiments, the moving target detection model is a behavior analysis model, which may be used to perform behavior analysis on the point cloud of the total scores to be merged to obtain output action categories, such as walking, sitting on a chair, standing, lying on the floor, lying on a bed, falling, etc.

[0090] The above embodiment is not limited to the configuration of the behavior analysis model, and may refer to related techniques.

[0091] In some embodiments, as shown in FIG. 11, the method further includes the following steps:

[0092] Step 1103: Post-processing is performed on the detection result of the moving target to obtain the processed moving target.

[0093] In some embodiments, performing post-processing on the moving target includes the following steps.

[0094] Step S31: Extract moving target detection results of the current frame and the immediately preceding n consecutive frames.

[0095] Step S32: In the post-processing model, the moving target detection results of the current frame and the immediately preceding consecutive n frames are temporally combined by 1D convolution.

[0096] Step S33: In the post-processing model, the moving targets are connected by full connection to obtain the moving target of the current frame after correction.

[0097] In the above embodiment, the detection accuracy can be further improved by performing post-processing on the detection result of the detected moving target (for example, the detection result of the human body keypoints).

[0098] Fig. 13 is a schematic diagram of an example of a post-processing model. As shown in Fig. 13, when the input multi-frame moving target detection results (human body keypoint detection results) are input into the post-processing model, the post-processing model temporally combines the moving target detection results of these frames through 1D convolution, and then performs combination between the moving targets through full combination to obtain the moving target in the current frame after correction.

[0099] The present invention is not limited to the configuration of the post-processing model, and may be modified based on FIG. 13, for example, some layer configurations may be added or some layer configurations may be deleted, and related technologies may be referred to.

[0100] Note that the above FIG. 11 merely illustrates an embodiment of the present invention, and the present invention is not limited thereto. For example, the execution order between each step may be adjusted as appropriate, other steps may be added, or some steps may be deleted. Those skilled in the art may make appropriate modifications based on the above content, and are not limited to the description of the above FIG. 11.

[0101] Table 1 is an example of a keypoint detection result after pre-processing the radar point cloud using the radar point cloud processing method according to the embodiment of the present invention.

[0102] [Table 1] As shown in Table 1, for person 117 (walking), the method in Fig. 9 had an error of only 0.0629, and the detection accuracy improved by 23.7%, while the method in Fig. 10 had an error of only 0.0628, and the detection accuracy improved by 23.8%. For person 128 (falling), the method in Fig. 9 had an error of only 0.0499, and the detection accuracy improved by 38.4%, while the method in Fig. 10 had an error of only 0.0499, and the detection accuracy improved by 38.4%.

[0103] Table 2 shows an example of a test result after post-processing is performed on the keypoint detection result using the post-processing method according to the embodiment of the present invention.

[0104] [Table 2] As shown in Table 2, for person 117 (walking), when post-processing was performed on the keypoint detection results of 10 frames, the error was only 0.0613, and the accuracy improved by 8.17%. For person 128 (falling), when post-processing was performed on the keypoint detection results of 10 frames, the error was only 0.0488, and the accuracy improved by 7.22%.

[0105] According to this embodiment, point cloud pre-processing is performed on the radar point clouds of multiple frames, with the radar frame as a unit, the radar frames to be point cloud-merged are determined in accordance with a point cloud merging policy, and the radar point clouds in the determined radar frames to be point cloud-merged are selected as input to a moving target detection model in accordance with a predetermined policy, thereby making it possible to detect moving targets and improve detection accuracy without the need to limit the category of motion.

[0106] <Example 3> The embodiment of the present invention provides a radar point cloud processing device for detecting moving targets, and the principle of problem solving is the same as that of the first embodiment, so the description of the similar contents will be omitted.

[0107] 14 is a schematic diagram of an example of a radar point cloud processing device according to an embodiment of the present invention. As shown in FIG. 14, a radar point cloud processing device 1400 according to an embodiment of the present invention includes the following units.

[0108] The pre-processing unit 1401 performs point cloud pre-processing on the radar point clouds of multiple frames, with each frame being a unit of radar frame.

[0109] The merging processor 1402 performs frame merging processing on multiple radar frames that have been subjected to point cloud pre-processing.

[0110] A post-processing unit 1403 performs point cloud post-processing on the radar point clouds in the multiple radar frames that have been subjected to the frame merging process.

[0111] In some embodiments, when performing point cloud pre-processing on the radar point cloud of a single frame, the pre-processing unit 1401 filters the point cloud in the current radar frame and determines the point cloud to be retained in the radar frame according to a predetermined policy.

[0112] In some embodiments, the predetermined policy is that the number of points in a radar frame does not exceed a first threshold.

[0113] In the above embodiment, if the number of the filtered point clouds does not exceed the first threshold, the pre-processing unit sets the filtered point clouds as the point clouds to be reserved in the radar frame, and if the number of the filtered point clouds exceeds the first threshold, the pre-processing unit samples the filtered point clouds to obtain the point clouds to be reserved in the radar frame, where the number of the reserved point clouds in the radar frame does not exceed the first threshold.

[0114] In some embodiments, the predetermined policy is that the number of points in a radar frame does not exceed a first threshold.

[0115] In the above embodiment, if the number of the filtered point clouds does not exceed a first threshold, the pre-processing unit sets the filtered point clouds as the point clouds to be reserved in the radar frame, and if the number of the filtered point clouds exceeds the first threshold, the pre-processing unit samples the filtered point clouds to obtain the point clouds to be reserved in the radar frame, where the number of the reserved point clouds in the radar frame does not exceed the first threshold.

[0116] In some embodiments, the predetermined policy is that the number of points in a radar frame does not exceed a predetermined percentage of the total number of points to be merged.

[0117] In the above embodiment, if the number of the filtered point clouds does not exceed a predetermined ratio of the total number of points to be merged, the pre-processing unit sets the filtered point clouds as the points to be retained in the radar frame, and if the number of the filtered point clouds exceeds a predetermined ratio of the total number of points to be merged, the pre-processing unit samples the filtered point clouds to obtain the points to be retained in the radar frame, where the number of the retained point clouds in the radar frame does not exceed a predetermined ratio of the total number of points to be merged.

[0118] In some embodiments, the merging processor 1402 performs a validity determination on the data of the current radar frame, and if the data of the radar frame is valid, merges the point clouds in each immediately preceding radar frame sequentially within a certain number of frames, starting from the radar frame, until the number of point clouds is at least equal to the total number of points to be merged.

[0119] In some embodiments, if the total number of point clouds in the radar frame and a predetermined number of radar frames immediately preceding the radar frame is less than a second threshold, the merging processing unit 1402 determines that the data in the radar frame is invalid, and otherwise determines that the data in the radar frame is valid.

[0120] In some embodiments, the post-processing unit 1403 selects the total number of point clouds to be merged based on the spatial distribution characteristics and / or velocity distribution characteristics of the point clouds in the multiple radar frames after merging.

[0121] In some embodiments, the post-processing unit 1403 samples all point clouds in the multiple radar frames after the merging process to obtain a point cloud of the total number of points to be merged, or the post-processing unit 1403 reserves point clouds in other radar frames other than the last frame in time among the multiple radar frames after the merging process, samples the point cloud in the last frame, and obtains a point cloud of the total number of points to be merged.

[0122] The embodiment of the present invention further provides a moving target detection device, and the principle of solving the problem is the same as that of the first embodiment, so the description of the similar contents will be omitted.

[0123] 15 is a schematic diagram of an example of a moving target detection device according to an embodiment of the present invention. As shown in FIG. 15, a moving target detection device 1500 according to an embodiment of the present invention includes the following devices.

[0124] The processing device 1501 performs point cloud pre-processing on the radar point clouds of multiple frames using the method of the first embodiment, and obtains a point cloud of the total number of points to be merged.

[0125] The detection device 1502 performs moving target detection on the point cloud of the total number of points to be merged using a moving target detection model, and acquires moving targets.

[0126] In the above embodiment, the processing device 1501 may be realized by the radar point cloud processing device 1400 according to the above embodiment, the contents of which are incorporated herein by reference, and the description thereof will be omitted here.

[0127] In the above embodiment, the detection device 1502 may perform keypoint detection on the input radar point cloud using the keypoint detection model shown in Fig. 12 to obtain a keypoint detection result. The present invention is not limited thereto, and the detection device 1502 may perform detection or analysis on the input radar point cloud using other similar keypoint detection models or behavior analysis models, etc., to obtain a detection result or analysis result.

[0128] In some embodiments, as shown in FIG. 15, the moving target detection apparatus 1500 may further include the following devices:

[0129] The post-processing device 1503 performs post-processing on moving target detection results (eg, keypoint detection results) of multiple frames using a post-processing model, and obtains final moving target (keypoint) detection results.

[0130] In the above embodiment, the post-processing device 1503 may perform post-processing on the moving target detection results (keypoint detection results) of multiple frames using the post-processing model shown in FIG. 13, but the present invention is not limited to this, and the post-processing device 1503 may perform post-processing on the moving target detection results (keypoint detection results) of multiple frames using other similar post-processing models.

[0131] It should be noted that the above merely describes each component or module related to the present invention, and the present invention is not limited thereto. The above-mentioned devices 1400 and 1500 may include other components or modules, and the specific contents of these components or modules may be referred to in the related art.

[0132] For simplicity, Figures 14 and 15 merely show the connection relationships or signal directions between each component or module as examples, and it is clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned various components or modules may be realized by hardware devices such as a processor and a memory, and the embodiment of the present invention is not limited thereto.

[0133] The above-mentioned embodiments are merely illustrative of the embodiments of the present invention, and the present invention is not limited thereto, and appropriate modifications may be made based on the above-mentioned various embodiments. For example, the above-mentioned embodiments may be used alone, or one or more of the above-mentioned embodiments may be used in combination.

[0134] According to this embodiment, point cloud pre-processing is performed on the radar point clouds of multiple frames, with the radar frame as a unit, the radar frames to be point cloud-merged are determined in accordance with a point cloud merging policy, and the radar point clouds in the determined radar frames to be point cloud-merged are selected as input to a moving target detection model in accordance with a predetermined policy, thereby making it possible to detect moving targets and improve detection accuracy without the need to limit the category of motion.

[0135] <Example 4> An embodiment of the present invention provides an electronic device including the apparatus 1400, 1500 described in the third embodiment, the contents of which are incorporated herein by reference. The electronic device may be, for example, a computer, a server, a workstation, a laptop computer, a smartphone, etc., but the embodiment of the present invention is not limited thereto.

[0136] Fig. 16 is a schematic diagram of an electronic device according to an embodiment of the present invention. As shown in Fig. 16, the electronic device 1600 according to the embodiment of the present invention includes a processor (e.g., a central processing unit (CPU)) 1610 and a memory 1620. The memory 1620 is connected to the processor 1610. The memory 1620 may store various data and may further store an information processing program 1621. The program 1621 is executed under the control of the processor 1610.

[0137] In some aspects, the functions of the devices 1400 and 1500 described in the third embodiment may be integrated into the processor 1610. Here, the processor 1610 may be configured to implement the method described in the first embodiment.

[0138] In some aspects, the devices 1400, 1500 described in Example 3 may each be configured with a processor 1610, for example, the devices 1400, 1500 described in Example 3 may be chips connected to the processor 1610 and configured to realize the functions of the devices 1400, 1500 described in Example 3 under the control of the processor 1610.

[0139] 16, the electronic device 1600 may further include an input / output (I / O) device 1630 and a display 1640. Here, the functions of the components are similar to those of the conventional technology, and the description thereof will be omitted here. Note that the electronic device 1600 does not need to include all the components shown in FIG. 16. The electronic device 1600 may also include components not shown in FIG. 16, and may refer to the conventional technology.

[0140] An embodiment of the present invention provides a computer-readable program that, when executed in an electronic device, causes a computer to execute the method described in embodiment 1 in the electronic device.

[0141] An embodiment of the present invention further provides a storage medium having stored thereon a computer-readable program for causing a computer to execute the method according to embodiment 1 in an electronic device.

[0142] The above-mentioned apparatus and method of the present invention may be realized by hardware or a combination of hardware and software.The present invention relates to a computer-readable program, which, when executed by a logic unit, causes the logic unit to realize the above-mentioned apparatus or components, or to realize the above-mentioned various methods or steps.The present invention relates to a storage medium for storing the above-mentioned program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0143] The methods / apparatuses described with reference to the embodiments of the present invention may be implemented in hardware, software modules executed by a processor, or a combination of both. For example, one or more of the functional block diagrams shown in the drawings, or one or more combinations of the functional block diagrams, may correspond to each software module in a computer program flow or each hardware module. These software modules may correspond to each step shown in the drawings. These hardware modules may be realized by implementing these software modules in hardware, for example, using a field programmable gate array (FPGA).

[0144] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium known to those skilled in the art. The storage medium may be connected to the processor so that the processor reads information from the storage medium and writes information to the storage medium, or the storage medium may be a component of the processor. The processor and the storage medium are located in an ASIC. The software module may be stored in the memory of the mobile terminal, or in a memory card inserted in the mobile terminal. For example, if the device (e.g., a mobile terminal) uses a MEGA-SIM card or a large capacity flash memory device with a relatively large capacity, the software module may be stored in the MEGA-SIM card or the large capacity flash memory device.

[0145] One or more of the functional blocks and / or one or more combinations of functional blocks depicted in the drawings may be implemented in a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any suitable combination thereof to perform the functions described herein. One or more of the functional blocks and / or one or more combinations of functional blocks depicted in the drawings may be implemented in, for example, a combination of computing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, one or more microprocessors in combination with a DSP communication, or any other configuration.

[0146] Although the present invention has been described above with reference to specific embodiments, the above description is merely illustrative and does not limit the scope of protection of the present invention. Various modifications and changes may be made to the present invention without departing from the spirit and principles of the present invention, and these modifications and changes also fall within the scope of the present invention.

[0147] Furthermore, the following supplementary notes are disclosed regarding the embodiments including the above-mentioned examples. (Appendix 1) 1. A method for processing radar point clouds for moving target detection, comprising the steps of: S1: performing point cloud pre-processing on radar point clouds of multiple frames, with each frame being a unit of radar frame; S2: performing frame merging processing on the multiple radar frames that have been subjected to point cloud pre-processing; S3: performing post-point cloud processing on the radar point clouds in the multiple radar frames that have been subjected to frame merging processing. (Appendix 2) In S1, the step of performing point cloud pre-processing on a radar point cloud of a single frame includes: S11: filtering a point cloud in a current radar frame; S12: determining a set of points to be retained in the radar frame according to a predetermined policy. (Appendix 3) In S11, 3. The method of claim 2, wherein the filtering includes at least one of spatial filtering, velocity filtering, and cluster filtering. (Appendix 4) In S12, the predetermined policy is 3. The method of claim 2, wherein the number of points in the radar frame does not exceed a first threshold. (Appendix 5) The S12 is If the number of the filtered points does not exceed the first threshold, the filtered points are set as a retained point set in the radar frame. and if the number of the filtered point clouds exceeds the first threshold, sampling the filtered point cloud to obtain a retained point cloud in the radar frame, wherein the number of the retained point clouds in the radar frame does not exceed the first threshold. (Appendix 6) In S12, the predetermined policy is 3. The method of claim 2, wherein the number of points in the radar frame does not exceed a predetermined percentage of the total number of points to be merged. (Appendix 7) The S12 is if the number of the filtered points does not exceed the predetermined percentage of the total number of points to be merged, setting the filtered points as a retained point set in the radar frame; and if the number of the filtered point clouds exceeds the predetermined percentage of the total number of points to be merged, sampling the filtered point clouds to obtain a retained point cloud in the radar frame, wherein the number of the retained point clouds in the radar frame does not exceed the predetermined percentage of the total number of points to be merged. (Appendix 8) S2 is S21: performing a validity determination on data of a current radar frame; S22: If the data of the radar frame is valid, sequentially merging the point clouds in each immediately preceding radar frame within a certain number of frames, starting from the radar frame, until the number of point clouds is at least equal to the total number of points to be merged. (Appendix 9) S21 is, 9. The method of claim 8, further comprising: determining that the data of the radar frame is invalid if the total number of point clouds of the radar frame and a predetermined number of radar frames immediately preceding the radar frame is less than a second threshold; and determining that the data of the radar frame is valid if not. (Appendix 10) S3 is The method of claim 1, further comprising a step of selecting a total number of point clouds to be merged based on the spatial distribution characteristics and / or velocity distribution characteristics of the point clouds in the multiple radar frames after the merging process. (Appendix 11) S3 is 11. The method of claim 10, further comprising: sampling all point clouds in multiple radar frames after merging processing to obtain a point cloud of the total number of points to be merged. (Appendix 12) S3 is a step of reserving point clouds in radar frames other than the last frame in time among the plurality of radar frames after the merging process; and sampling the point cloud in the final frame to obtain a point cloud of the total number of points to be merged. (Appendix 13) A method for detecting a moving target, comprising: Performing point cloud pre-processing on radar point clouds of multiple frames using a method according to any one of appendices 1 to 12 to obtain a point cloud of a total number of points to be merged; and performing moving target detection on the point cloud of the total number of points to be merged using a moving target detection model to obtain key points of the human body. (Appendix 14) Extracting moving target detection results of a current frame and the immediately preceding n consecutive frames; In a post-processing model, temporally combining the moving target detection results of the current frame and the immediately preceding n consecutive frames by 1D convolution; The method of claim 13, further comprising a step of performing full-connection coupling between moving targets in the post-processing model to obtain a corrected moving target of the current frame. (Appendix 15) 15. An electronic device comprising: a memory having a computer program stored therein; and a processor, the processor configured to execute the computer program to implement a method according to any one of claims 1 to 14. (Appendix 16) A storage medium having a computer readable program stored thereon, the computer readable program causing a computer to execute a method according to any one of claims 1 to 14 in an electronic device.

Claims

1. 1. An apparatus for processing radar point clouds for moving target detection, comprising: a pre-processing unit that performs point cloud pre-processing on radar point clouds of a plurality of frames, each of the frames being a unit of radar frame; A frame merging processing unit that performs a frame merging process on a plurality of radar frames that have been subjected to point cloud pre-processing; a post-processing unit that performs point cloud post-processing on the radar point clouds in the plurality of radar frames that have been subjected to frame merging processing.

2. When performing point cloud pre-processing on a radar point cloud of a single frame, the pre-processing unit Filtering the point cloud in the current radar frame; The apparatus of claim 1 , further comprising: determining which points in the radar frame to withhold according to a predetermined policy.

3. The predetermined policy is: a number of points in the radar frame does not exceed a first threshold; If the number of the point cloud after filtering does not exceed the first threshold, the preprocessing unit determines the point cloud after filtering as a point cloud to be retained in the radar frame; 3. The apparatus of claim 2, wherein if the number of the filtered point clouds exceeds the first threshold, the preprocessing unit samples the filtered point clouds to obtain a retained point cloud in the radar frame, and the number of the retained point clouds in the radar frame does not exceed the first threshold.

4. The predetermined policy is: the number of points in the radar frame does not exceed a predetermined percentage of the total number of points to be merged; If the number of the points after filtering does not exceed the predetermined ratio of the total number of points to be merged, the preprocessing unit sets the points after filtering as a point group to be retained in the radar frame; 3. The apparatus of claim 2, wherein if the number of the filtered point cloud exceeds the predetermined percentage of the total number of points to be merged, the pre-processing unit samples the filtered point cloud to obtain a retained point cloud in the radar frame, and the number of the retained point cloud in the radar frame does not exceed the predetermined percentage of the total number of points to be merged.

5. The merging processing unit includes: Performs a validity check on the data in the current radar frame, 2. The apparatus of claim 1, further comprising: starting with the radar frame, if the data for the radar frame is valid, sequentially merging the point clouds in each immediately preceding radar frame within a certain number of frames until the number of point clouds is at least equal to the total number of points to be merged.

6. 6. The device of claim 5, wherein if a total number of point clouds of the radar frame and a predetermined number of radar frames immediately preceding the radar frame is less than a second threshold, the merging processing unit determines that the data of the radar frame is invalid, and otherwise determines that the data of the radar frame is valid.

7. The apparatus of claim 1 , wherein the post-processing unit selects a total number of point clouds to be merged based on spatial distribution characteristics and / or velocity distribution characteristics of the point clouds in the multiple radar frames after merging processing.

8. The post-processing unit samples all point clouds in the multiple radar frames after the merging process to obtain the point cloud of the total number of points to be merged, or The apparatus of claim 7, wherein the post-processing unit reserves point clouds in radar frames other than the last frame in time among the multiple radar frames after the merging process, samples the point cloud in the last frame, and obtains the point cloud of the total number of points to be merged.

9. A moving target detection device, comprising: The apparatus according to any one of claims 1 to 8, wherein a point cloud pre-processing is performed on radar point clouds of a plurality of frames to obtain a point cloud of a total number of points to be merged; and a detection device that performs moving target detection on the point cloud of the total number of points to be merged using a moving target detection model, and acquires moving targets.

10. An electronic device including a memory in which a computer program is stored and a processor, the processor comprising: performing point cloud pre-processing on radar point clouds of a plurality of frames, each of the frames being a unit of radar frame; performing a frame merging process on the plurality of radar frames that have been subjected to point cloud pre-processing; performing point cloud post-processing on the radar point clouds in the plurality of radar frames that have been subjected to the frame merging process to obtain a point cloud of a total number of points to be merged; and performing moving target detection on the point cloud of the total number of points to be merged using a moving target detection model to obtain a moving target.

Images