Millimeter-Wave Radar-Based Target Recognition Method and Terminal Device

Abstract

The present disclosure provides a target recognition method and a terminal device based on a millimeter-wave radar. The method includes: for any frame of frame data collected by the millimeter-wave radar, determining the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar; clustering each target point by using the determined position coordinates of each target point to obtain at least one target area; for any target area, performing rasterization processing on the target area by using the position coordinates of each target point in the target area and the speed of each target point determined from the frame data to obtain a raster image; inputting the raster image into a preset target recognition model to determine the target category and the target action corresponding to the target area; thereby, the present disclosure only needs one frame of data to recognize the target, without the accumulation of multiple frames, so as to reduce the delay of target recognition and improve the target recognition efficiency.

Description

Background Art

[0002] With the rapid development of the intelligence of vision technology, more and more fields are trying to integrate vision technology. However, vision technology is not universal in any field. For example, in the transportation field, in bad weather or poor lighting conditions, the performance of vision technology will be greatly reduced. Or in the home field, the introduction of vision technology will violate people's privacy issues.

[0003] The all-weather working conditions of millimeter-wave radar and the information mode of non-RGB can well solve the above problems. Millimeter-wave radar can work all-weather, without being restricted by weather and light conditions. It does not require RGB information and can well protect the privacy of users in the home environment.

[0004] In the prior art, millimeter-wave radar performs target recognition based on micro-Doppler feature extraction. This method requires the accumulation of multiple frames, resulting in a certain delay in target recognition and reducing the recognition efficiency. Summary of the Invention

[0005] In an exemplary embodiment of the present disclosure, a target recognition method and a terminal device based on millimeter-wave radar are provided to reduce the delay in target recognition and thereby improve the target recognition efficiency.

[0006] A first aspect of the present disclosure provides a terminal device, including a processor and a millimeter-wave radar;

[0007] The millimeter-wave radar is configured to collect frame data;

[0008] The processor is configured to:

[0009] For any frame of the frame data collected by the millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar;

[0010] Cluster each target point by using the determined position coordinates of each target point to obtain at least one target area;

[0011] For any target area, perform rasterization processing on the target area by using the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to obtain a raster image;

[0012] Input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0013] In this embodiment, the position coordinates of each target point are determined from a frame of data, and clustering is performed using the position coordinates of each target point to obtain multiple target regions. By inputting the rasterized target regions into the target recognition model, the target category and actions are determined. Thus, the embodiments of the present disclosure can identify the target with only one frame of data without the accumulation of multiple frames, reducing the latency of target recognition and improving the efficiency of target recognition.

[0014] In one embodiment, when the processor executes rasterizing the target region using the position coordinates of each target point in the target region and the velocity of each target point determined from the frame data to obtain a raster image, it is specifically configured to:

[0015] Divide the target region into raster blocks of a specified size;

[0016] Determine the relative position coordinates of each target point through the position coordinates of the target point with the smallest distance from the millimeter-wave radar in the target region and the position coordinates of each target point;

[0017] Determine the raster block where each target point is located according to the relative position coordinates of each target point and the position of each raster block in the target region;

[0018] Use the preset correspondence between the velocity of each target point and the gray value to determine the gray value corresponding to the velocity of each target point;

[0019] Set the gray value of the raster block according to the gray value of the target point included in the raster block, and set the gray value of each raster block that does not contain a target point to a specified gray value, where the specified gray value is different from the gray value of the raster block containing the target point.

[0020] In this embodiment, the target region is rasterized through the position coordinates and velocity of the target point, making the result of target recognition more accurate.

[0021] In one embodiment, when the processor executes setting the gray value of the raster block according to the gray value of the target point included in the raster block, it is specifically configured to:

[0022] If the number of target points included in the raster block is one target point, set the gray value of the raster block to the gray value corresponding to the target point; and,

[0023] If the number of target points included in the raster block is multiple target points, set the gray value of the raster block to the average value of the gray values corresponding to each target point respectively.

[0024] In this embodiment, the gray value of the grid block is determined by the number of target points included in the grid block, so as to make the determination of the gray value more accurate.

[0025] In one embodiment, after the processor clusters each target point by using the determined position coordinates of each target point to obtain at least one target area, the processor is further configured to:

[0026] Determine the position coordinates of the target area according to the position coordinates of each target point in each target area; and,

[0027] Determine the speed of the target area by the speeds of each target point in each target area.

[0028] In this embodiment, the position coordinates and speed of the target area are respectively determined by the position coordinates and speed of each target point.

[0029] In one embodiment, when the processor determines the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar, the processor is specifically configured to:

[0030] Use the determined distance between each target point and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar to respectively determine the abscissa and ordinate of each target point; and,

[0031] Determine the vertical coordinate of each target point according to the distance between each target point and the millimeter-wave radar and the elevation angle.

[0032] In this embodiment, the position coordinates of the target point are determined by determining the abscissa, ordinate and vertical coordinate of each target point.

[0033] In one embodiment, the processor is further configured to:

[0034] Train the target recognition model according to the following method:

[0035] Input the target recognition training sample into the target recognition model, extract features of the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a grid image and a labeled target category, and the labeled target category includes a target category and a target action;

[0036] Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value;

[0037] When the error value does not meet the specified conditions, after adjusting the training parameters of the target recognition model, return to execute the step of inputting the target recognition training samples into the target recognition model until the error value meets the specified conditions, and then end the training of the target recognition model.

[0038] In this embodiment, the training samples are input into the target recognition model for training to determine the error value, and the training parameters of the target recognition model are adjusted through the error value until the error value meets the specification, and then the training of the target recognition model is ended.

[0039] The second aspect of the present disclosure provides a target recognition method based on a millimeter-wave radar, and the method includes:

[0040] For any frame of frame data collected by the millimeter-wave radar, according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar, determine the position coordinates of each target point;

[0041] Cluster each target point by using the determined position coordinates of each target point to obtain at least one target area;

[0042] For any target area, use the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to perform rasterization processing on the target area to obtain a raster image;

[0043] Input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0044] In one embodiment, the using the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to perform rasterization processing on the target area to obtain a raster image includes:

[0045] Divide the target area into grid blocks of a specified size;

[0046] Determine the relative position coordinates of each target point through the position coordinates of the target point with the smallest distance from the millimeter-wave radar in the target area and the position coordinates of each target point;

[0047] Determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target area;

[0048] Use the preset correspondence between the speed of each target point and the gray value to determine the gray value corresponding to the speed of each target point;

[0049] Set the gray value of the grid block according to the gray value of the target point included in the grid block, and set the gray values of the grid blocks that do not contain target points to a specified gray value, where the specified gray value is different from the gray value of the grid block containing the target point.

[0050] In one embodiment, when the processor executes setting the gray value of the grid block according to the gray value of the target point included in the grid block, it is specifically configured to:

[0051] If the number of target points included in the grid block is one target point, set the gray value of the grid block to the gray value corresponding to the target point; and,

[0052] If the number of target points included in the grid block is multiple target points, set the gray value of the grid block to the average value of the gray values respectively corresponding to each target point.

[0053] In one embodiment, after clustering each target point by using the determined position coordinates of each target point to obtain at least one target area, the method further includes:

[0054] Determine the position coordinates of the target area according to the position coordinates of each target point in each target area; and,

[0055] Determine the speed of the target area through the speeds of each target point in each target area.

[0056] In one embodiment, determining the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar includes:

[0057] Use the determined distance between each target point and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar to respectively determine the abscissa and ordinate of each target point; and,

[0058] Determine the vertical coordinate of each target point according to the distance between each target point and the millimeter-wave radar and the pitch angle.

[0059] In one embodiment, train the target recognition model according to the following method:

[0060] Input the target recognition training sample into the target recognition model, extract features of the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a grid image and a labeled target category, and the labeled target category includes a target category and a target action.

[0061] Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value;

[0062] When the error value does not meet the specified condition, after adjusting the training parameters of the target recognition model, return to execute the step of inputting the target recognition training sample into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

[0063] According to the third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0064] At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executed by the at least one processor; the instructions are executed by the at least one processor so that the at least one processor can execute the method as described in the second aspect.

[0065] According to the fourth aspect provided by the embodiments of the present disclosure, there is provided a computer storage medium, and the computer storage medium stores a computer program for executing the method as described in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0067] Figure 1 It is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure;

[0068] Figure 2 It is one of the flowcharts of a target recognition method based on a millimeter-wave radar according to an embodiment of the present disclosure;

[0069] Figure 3 It is one of the flowcharts of a rasterization process according to an embodiment of the present disclosure;

[0070] Figure 4 It is a schematic diagram of rasterization processing according to an embodiment of the present disclosure;

[0071] Figure 5 It is another flowchart of a target recognition method based on a millimeter-wave radar according to an embodiment of the present disclosure;

[0072] Figure 6 It is a target recognition device based on a millimeter-wave radar according to an embodiment of the present disclosure;

[0073] Figure 7 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some but not all of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.

[0075] In the embodiments of the present disclosure, the term "and / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0076] The application scenarios described in the embodiments of the present disclosure are for more clearly illustrating the technical solutions in the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided in the embodiments of the present disclosure. Those of ordinary skill in the art know that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present disclosure are equally applicable to similar technical problems. Among them, in the description of the present disclosure, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0077] In the prior art, millimeter-wave radar performs target recognition based on micro-Doppler feature extraction. This method requires the accumulation of multiple frames, resulting in a certain delay in target recognition and reducing the recognition efficiency.

[0078] Therefore, the present disclosure provides a target recognition method for millimeter-wave radar. By determining the position coordinates of each target point from one frame of data and clustering using the position coordinates of each target point to obtain multiple target regions, and inputting the rasterized target regions into a target recognition model to determine the target category and action. Thus, the present disclosure can recognize the target with only one frame of data and does not require the accumulation of multiple frames, so the delay in target recognition is reduced and the target recognition efficiency is improved.

[0079] Before introducing the solution of the present disclosure in detail, first, the structure of the terminal device in the embodiments of the present disclosure will be introduced. Figure 1 A schematic structural diagram of the terminal device in the present disclosure. As Figure 1As shown in the figure, the terminal device in the embodiments of the present disclosure includes: a processor 110 and a millimeter-wave radar 120. Among them, the millimeter-wave radar is used to collect frame data; the processor 110 is used to determine the position coordinates of each target point for any frame of the frame data collected by the millimeter-wave radar 120 according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar; the processor 110 clusters each target point by using the determined position coordinates of each target point to obtain at least one target area; for any target area, rasterize the target area by using the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to obtain a raster image; and input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0080] In one embodiment, when the processor 110 executes the rasterization process of the target area by using the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to obtain a raster image, it is specifically configured as follows:

[0081] Divide the target area into grid blocks of a specified size;

[0082] Determine the relative position coordinates of each target point through the position coordinates of the target point with the smallest distance from the millimeter-wave radar in the target area and the position coordinates of each target point;

[0083] Determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target area;

[0084] Use the preset correspondence between the speed of each target point and the gray value to determine the gray value corresponding to the speed of each target point;

[0085] Set the gray value of the grid block according to the gray value of the target point included in the grid block, and set the gray value of each grid block that does not include a target point to a specified gray value, where the specified gray value is different from the gray value of the grid block including the target point.

[0086] In one embodiment, when the processor 110 executes the setting of the gray value of the grid block according to the gray value of the target point included in the grid block, it is specifically configured as follows:

[0087] If the number of target points included in the grid block is one target point, set the gray value of the grid block to the gray value corresponding to the target point; and,

[0088] If the number of target points included in the grid block is multiple target points, set the gray value of the grid block to the average of the gray values respectively corresponding to each target point.

[0089] In one embodiment, after the processor 110 performs clustering on each target point by using the position coordinates of each determined target point to obtain at least one target area, it is further configured to:

[0090] Determine the position coordinates of the target area according to the position coordinates of each target point in each target area; and,

[0091] Determine the speed of the target area by the speeds of each target point in each target area.

[0092] In one embodiment, when the processor 110 determines the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar, it is specifically configured to:

[0093] Use the determined distance between each target point and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar to respectively determine the abscissa and ordinate of each target point; and,

[0094] Determine the vertical coordinate of each target point according to the distance between each target point and the millimeter-wave radar and the pitch angle.

[0095] In one embodiment, the processor 110 is further configured to:

[0096] Train the target recognition model according to the following method:

[0097] Input the target recognition training sample into the target recognition model, extract features from the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a grid image and a labeled target category, and the labeled target category includes a target category and a target action;

[0098] Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value;

[0099] When the error value does not meet the specified condition, adjust the training parameters of the target recognition model, and then return to execute the step of inputting the target recognition training sample into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

[0100] After introducing the terminal device according to the embodiments of the present disclosure, the technical solutions of the present disclosure will be introduced in detail.

[0101] Figure 2 As shown in the flowchart of the method for target recognition based on millimeter-wave radar according to the present disclosure, the method may include the following steps:

[0102] Step 201: For any frame of frame data collected by the millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar;

[0103] It should be noted that: in the embodiments of the present disclosure, the target point is the reflection point on the target object that reflects the electromagnetic wave after the millimeter-wave radar emits the electromagnetic wave to the target object. And the frame data in the embodiments of the present disclosure is the intermediate-frequency signal of the millimeter-wave radar after ADC sampling.

[0104] Among them:

[0105] a. The method for determining the distance between each target point and the millimeter-wave radar is: perform Fourier transform on the number of chirp signals in the frame data to determine the frequency, and then determine the distance between each target point and the millimeter-wave radar through formula (1):

[0106]

[0107] Where d is the distance between the target point and the millimeter-wave radar, f is the frequency, c is the speed of light, and s is the slope of the chirp signal.

[0108] b. The method for determining the speed of each target point is: perform Fourier transform on the number of samples of each chirp signal to obtain the phase. Then determine the speed of each target point through formula (2):

[0109]

[0110] Where λ is the wavelength corresponding to the starting frequency of the millimeter-wave radar, ω is the phase, and T C is the interval time between two chirp signals.

[0111] c. The method for determining the azimuth angle formed by each target point and the millimeter-wave radar is: perform Fourier transform on the number of antennas in the horizontal direction and the number of antennas in the vertical direction of each chirp signal respectively to obtain the horizontal phase and the vertical phase; then determine the azimuth angle formed by each target point and the millimeter-wave radar through formula (3), and determine the pitch angle formed by each target point and the millimeter-wave radar through formula (4):

[0112]

[0113] where θ is the azimuth angle formed by the target point and the millimeter-wave radar, and ω 1 is the horizontal phase, and d is the number of antennas of the millimeter-wave radar.

[0114]

[0115] where is the elevation angle formed by the target point and the millimeter-wave radar, and ω 2 is the vertical phase.

[0116] In one embodiment, step 202 described above may be implemented as: respectively determining the abscissa and ordinate of each target point by using the distance between each determined target point and the millimeter-wave radar, the azimuth angle formed by each target point and the millimeter-wave radar, and the elevation angle; determining the vertical coordinate of each target point according to the distance between each target point and the millimeter-wave radar and the elevation angle.

[0117] where the abscissa, ordinate, and vertical coordinate of each target point can be determined by formula (5):

[0118]

[0119] where p x is the abscissa of the target point, p y is the ordinate of the target point, p z is the vertical coordinate of the target point, where r is the distance between the target point and the millimeter-wave radar, θ is the azimuth angle formed by the target point and the millimeter-wave radar, is the elevation angle formed by the target point and the millimeter-wave radar.

[0120] Step 202: Clustering each target point by using the determined position coordinates of each target point to obtain at least one target area;

[0121] where the clustering algorithm used in the embodiments of the present disclosure is the density clustering algorithm.

[0122] Taking the DBSCAN algorithm as an example to explain the clustering process: For any target point, draw a circle with this target point as the center and a preset radius. If the total number of target points within this circle is less than the specified threshold, then determine this point as a boundary point. If the total number of target points within this circle is not less than the specified threshold, then determine this point as a core point. For any core point, determine each density-reachable point of this core point, and the area formed by the density-reachable points of this core point is the target area. In this way, each target area is determined.

[0123] It should be noted that the density clustering algorithm in the embodiments of the present disclosure includes, but is not limited to, the DBSCAN algorithm. The above embodiments of the present disclosure are only for explanation and do not limit the present disclosure.

[0124] Step 203: For any target area, rasterize the target area by using the position coordinates of each target point in the target area and the speed of each target point determined from the frame data to obtain a raster image.

[0125] In one embodiment, step 203 can be implemented as follows: As Figure 3 shown, the following steps may be included:

[0126] Step 301: Divide the target area into grid blocks of a specified size.

[0127] Step 302: Determine the relative position coordinates of each target point by using the position coordinate of the target point with the smallest distance from the millimeter-wave radar in the target area and the position coordinates of each target point.

[0128] Step 303: Determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target area.

[0129] Step 304: Determine the gray value corresponding to the speed of each target point by using the corresponding relationship between the speed and the gray value of each preset target point.

[0130] Step 305: Set the gray value of the grid block according to the gray value of the target point included in the grid block, and set the gray value of each grid block that does not contain a target point to a specified gray value, where the specified gray value is different from the gray value of the grid block containing the target point.

[0131] For example, as Figure 4 shown, there are two target areas, namely target area 1 and target area 2. Figure 4 The grid blocks in are the raster images after rasterizing target area 1 and target area 2 respectively. Among them, the gray value of the grid block corresponds to the speed of the target point included in the grid block. Among them, the corresponding relationship between the gray value and the speed of the target point can be shown in Table 1:

[0132] Table 1:

[0133] Velocity of the target point Gray value A to B (excluding B) m B to C n C to D p … …

[0134] In one embodiment, step 305 can be implemented as follows: if the number of target points included in the grid block is one target point, set the gray value of the grid block to the gray value corresponding to the target point; and if the number of target points included in the grid block is multiple target points, set the gray value of the grid block to the average value of the gray values respectively corresponding to each target point.

[0135] For example, if only target point 1 is included in grid block 1, and if the gray value corresponding to target point 1 is a, then set the gray value of grid block 1 to a. If the target points included in grid block 2 include target point 2 and target point 3, and if the gray value corresponding to target point 2 is b and the gray value corresponding to target point 3 is c. Then set the gray value of grid block 2 to

[0136] It should be noted that the size of the grid block can be set according to the actual situation. It can be set to a size that can only contain one target point, or the size of the grid block can be set to a size that contains multiple grid blocks. The present disclosure does not limit this here.

[0137] Among them, the larger the size of the grid block is set, the smaller the calculation amount is, but the quality of the obtained grid image is lower. The smaller the size of the grid block is set, the larger the calculation amount is, but the quality of the obtained grid image is higher.

[0138] Step 204: Input the grid image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0139] For example, the output results can be a person walking, a dog running, etc.

[0140] In one embodiment, train the target recognition model according to the following method:

[0141] Input the target recognition training samples into the target recognition model, extract features from the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training samples include grid images and labeled target categories, and the labeled target categories include target categories and target actions; compare the target category feature with the highest matching degree with the labeled target category to obtain an error value; when the error value does not meet the specified condition, adjust the training parameters of the target recognition model and then return to execute the step of inputting the target recognition training samples into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

[0142] In one embodiment, to make the information for target recognition more comprehensive, the position coordinates of the target area are determined based on the position coordinates of each target point in each target area; and the speed of the target area is determined by the speeds of each target point in each target area.

[0143] Among them, the average value of the position coordinates of each target point in each target area can be used as the position coordinates of the target area. And the average value of the speeds of each target point in each target area can be used as the speed of the target area.

[0144] For example, if the target points in the target area include target point 1, target point 2, target point 3, and target point 4. If the position coordinates of target point 1 are (10, 15, 20), the position coordinates of target point 2 are (15, 13, 18), the position coordinates of target point 3 are (17, 17, 19), and the position coordinates of target point 4 are (14, 15, 11), then the position coordinates of the target area can be determined as (14, 15, 17). If the speed of target point 1 is 1 m / s, the speed of target point 2 is 1.5 m / s, the speed of target point 3 is 1 m / s, and the speed of target point 4 is 1.5 m / s, then the speed of the target area is determined to be 1.25 m / s.

[0145] Thus, when this method is applied to a vehicle, the user can control the speed of the vehicle based on the speed and position of the object.

[0146] To further understand the technical solution of the present disclosure, the following is combined with Figure 5 for a detailed description, which may include the following steps:

[0147] Step 501: For any frame of data collected by the millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar;

[0148] Step 502: Cluster each target point using the determined position coordinates of each target point to obtain at least one target area;

[0149] Step 503: Determine the position coordinates of the target area according to the position coordinates of each target point in each target area; and determine the speed of the target area by the speeds of each target point in each target area;

[0150] Among them, the execution order between step 503 and step 504 is not limited in the present disclosure. Step 503 can be executed first, and then step 504; or step 504 can be executed first, and then step 503; or step 503 and step 504 can be executed simultaneously.

[0151] Step 504: Divide the target area into grid blocks of a specified size;

[0152] Step 505: Determine the relative position coordinates of each target point based on the position coordinates of the target point with the minimum distance from the millimeter-wave radar in the target area and the position coordinates of each target point;

[0153] Step 506: Determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target area;

[0154] Step 507: Use the preset correspondence between the speed and gray value of each target point to determine the gray value corresponding to the speed of each target point;

[0155] Step 508: Set the gray value of the grid block according to the gray value of the target points included in the grid block, and set the gray value of each grid block that does not contain target points to a specified gray value, where the specified gray value is different from the gray value of the grid block containing target points;

[0156] Step 509: For any target area, perform rasterization processing on the target area by using the position coordinates of each target point in the target area and the speed of each target point determined from the frame data to obtain a raster image;

[0157] Step 510: Input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0158] Based on the same inventive concept, the above-described millimeter-wave radar-based target recognition method of the present disclosure can also be implemented by a millimeter-wave radar-based target recognition device. The effect of the millimeter-wave radar-based target recognition is similar to that of the foregoing method, and will not be elaborated here.

[0159] Figure 6 FIG. is a schematic structural diagram of a millimeter-wave radar-based target recognition device according to an embodiment of the present disclosure.

[0160] As Figure 6 shown, the millimeter-wave radar-based target recognition device 600 of the present disclosure may include a target point position coordinate determination module 610, a target area determination module 620, a rasterization processing module 630, and a target recognition module 640.

[0161] The target point position coordinate determination module 610 is configured to, for any frame of data collected by a millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar determined from the frame data;

[0162] A target area determination module 620, configured to cluster each target point by using the position coordinates of each determined target point to obtain at least one target area;

[0163] A rasterization processing module 630, configured to perform rasterization processing on any target area by using the position coordinates of each target point in the target area and the speed of each target point determined from the frame data to obtain a raster image;

[0164] A target recognition module 640, configured to input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area.

[0165] In one embodiment, the rasterization processing module 630 is specifically configured to:

[0166] A segmentation unit 631, configured to segment the target area into raster blocks of a specified size;

[0167] A relative position coordinate determination unit 632, configured to determine the relative position coordinates of each target point by using the position coordinate of the target point with the minimum distance from the millimeter-wave radar in the target area and the position coordinates of each target point;

[0168] A raster block determination unit 633 for the target point, configured to determine the raster block where each target point is located according to the relative position coordinates of each target point and the position of each raster block in the target area;

[0169] A target point gray value determination unit 634, configured to determine the gray value corresponding to the speed of each target point by using the preset correspondence between the speed and gray value of each target point;

[0170] A raster block gray value setting unit 635, configured to set the gray value of the raster block according to the gray value of the target point included in the raster block, and set the gray value of each raster block not including a target point to a specified gray value, where the specified gray value is different from the gray value of the raster block including a target point.

[0171] In one embodiment, the raster block gray value setting unit 635 is specifically configured to:

[0172] If the number of target points included in the raster block is one target point, set the gray value of the raster block to the gray value corresponding to the target point; and,

[0173] If the number of target points included in the raster block is multiple target points, set the gray value of the raster block to the average value of the gray values respectively corresponding to each target point.

[0174] In one embodiment, the device further includes:

[0175] A target position determination module 650, configured to determine the position coordinates of the target area according to the position coordinates of each target point in each target area;

[0176] A target speed determination module 660, configured to determine the speed of the target area by the speeds of each target point in each target area.

[0177] In one embodiment, the target point position coordinate determination module 610 is specifically configured to:

[0178] Use the determined distances between each target point and the millimeter-wave radar, the azimuth angles and elevation angles formed by each target point and the millimeter-wave radar to respectively determine the abscissa and ordinate of each target point; and,

[0179] Determine the vertical coordinates of each target point according to the distances between each target point and the millimeter-wave radar and the elevation angles.

[0180] In one embodiment, the device further includes:

[0181] A target recognition model training module 670, configured to train the target recognition model according to the following method:

[0182] Input a target recognition training sample into the target recognition model, extract features of the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a grid image and a labeled target category, and the labeled target category includes a target category and a target action;

[0183] Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value;

[0184] When the error value does not meet the specified condition, adjust the training parameters of the target recognition model, and then return to execute the step of inputting the target recognition training sample into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

[0185] After introducing a target recognition method and a terminal device based on a millimeter-wave radar according to an exemplary embodiment of the present disclosure, next, an electronic device according to another exemplary embodiment of the present disclosure will be introduced.

[0186] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuits", "modules", or "systems" here.

[0187] In some possible implementation manners, an electronic device according to the present disclosure may at least include at least one processor and at least one computer storage medium. Among them, the computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps in the method for target recognition based on a millimeter-wave radar according to various exemplary implementation manners of the present disclosure described above in this specification. For example, the processor may execute steps such as Figure 2 shown in step 201 - 204.

[0188] The following refers to Figure 7 to describe the electronic device 700 according to this implementation manner of the present disclosure. Figure 7 The shown electronic device 700 is only an example and should not bring any limitation to the functions and usage scopes of the embodiments of the present disclosure.

[0189] As Figure 7 shown, the electronic device 700 is presented in the form of a general electronic device. The components of the electronic device 700 may include but are not limited to: the above-mentioned at least one processor 701, the above-mentioned at least one computer storage medium 702, and a bus 703 connecting different system components (including the computer storage medium 702 and the processor 701).

[0190] The bus 703 represents one or more of several types of bus structures, including a computer storage medium bus or a computer storage medium controller, a peripheral bus, a processor, or a local bus using any bus structure in a variety of bus structures.

[0191] The computer storage medium 702 may include a readable medium in the form of a volatile computer storage medium, such as a random access computer storage medium (RAM) 721 and / or a cache storage medium 722, and may further include a read-only computer storage medium (ROM) 723.

[0192] The computer storage medium 702 may also include a program / utility 725 having a set (at least one) of program modules 724. Such program modules 724 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0193] The electronic device 700 can also communicate with one or more external devices 704 (such as a keyboard, a pointing device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other electronic devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 705. Moreover, the electronic device 700 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 706. As shown in the figure, the network adapter 706 communicates with other modules for the electronic device 700 through the bus 703. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 700, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0194] In some possible implementation manners, various aspects of a target recognition method based on a millimeter-wave radar provided by the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps in the target recognition method based on the millimeter-wave radar according to various exemplary implementation manners of the present disclosure described above in this specification.

[0195] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access computer storage medium (RAM), a read-only computer storage medium (ROM), an erasable programmable read-only computer storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only computer storage medium (CD-ROM), an optical computer storage medium, a magnetic computer storage medium, or any suitable combination of the above.

[0196] The program product of the target recognition based on the millimeter-wave radar in the implementation manner of the present disclosure can adopt a portable compact disk read-only computer storage medium (CD-ROM) and include program code, and can run on an electronic device. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0197] A readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal can take various forms, including - but not limited to - electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0198] The program code contained on the readable medium can be transmitted using any appropriate medium, including - but not limited to - wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0199] The program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages - such as Java, C++, etc., and also including conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's electronic device, partially on the user's device, executed as a stand-alone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user's electronic device through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, can be connected to an external electronic device (e.g., by connecting through the Internet using an Internet service provider).

[0200] It should be noted that although several modules of the apparatus are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0201] In addition, although the operations of the method of the present disclosure are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the shown operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0202] Those skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic computer storage media, CD-ROM, optical computer storage media, etc.) that contain computer-usable program code.

[0203] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0204] These computer program instructions can also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable computer storage medium generate a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or the combination of blocks.

[0206] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure is also intended to include these changes and modifications.

Claims

1. A terminal device, characterized in that, it includes a processor and a millimeter-wave radar; the millimeter-wave radar is configured to collect frame data; the processor is configured to: For any frame of the frame data collected by the millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point determined by the frame data and the millimeter-wave radar, the azimuth angle and the pitch angle formed by each target point and the millimeter-wave radar; Cluster each target point by using the determined position coordinates of each target point to obtain at least one target area; For any target area, divide the target area into grid blocks of a specified size; determine the relative position coordinates of each target point through the position coordinates of the target point with the smallest distance from the millimeter-wave radar in the target area and the position coordinates of each target point; determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target area; use the preset correspondence between the speed and the gray value of each target point to determine the gray value corresponding to the speed of each target point; set the gray value of the grid block according to the gray value of the target point included in the grid block, and set the gray value of each grid block that does not include a target point to a specified gray value, where the specified gray value is different from the gray value of the grid block including the target point; Perform rasterization processing on the target area by using the position coordinates of each target point in the target area and the speed of each target point determined by the frame data to obtain a raster image; Input the raster image into a preset target recognition model to determine the target category and target action corresponding to the target area. Among them, the target recognition model is trained according to the following method: Input the target recognition training sample into the target recognition model, extract the target feature information from the raster image, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a raster image and a labeled target category, and the labeled target category includes a target category and a target action; Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value; When the error value does not meet the specified condition, after adjusting the training parameters of the target recognition model, return to execute the step of inputting the target recognition training sample into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

2. The terminal device according to claim 1, characterized in that, when the processor executes setting the gray value of the grid block according to the gray value of the target point included in the grid block, it is specifically configured to: if the number of target points included in the grid block is one target point, set the gray value of the grid block to the gray value corresponding to the target point; and if the number of target points included in the grid block is multiple target points, set the gray value of the grid block to the average value of the gray values corresponding to each target point respectively.

3. The terminal device according to claim 1, characterized in that, After the processor performs clustering on each target point by using the position coordinates of the determined target points to obtain at least one target region, it is further configured to: Determine the position coordinates of the target region according to the position coordinates of each target point in each target region; and, Determine the speed of the target region based on the speeds of the target points in each target region.

4. The terminal device according to claim 1, wherein, When the processor determines the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar, it is specifically configured to: Use the determined distance between each target point and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar to respectively determine the abscissa and ordinate of each target point; And, Determine the vertical coordinate of each target point according to the distance between each target point and the millimeter-wave radar and the elevation angle.

5. A target recognition method based on a millimeter-wave radar, wherein, The method includes: For any frame of frame data collected by a millimeter-wave radar, determine the position coordinates of each target point according to the distance between each target point determined from the frame data and the millimeter-wave radar, the azimuth angle and the elevation angle formed by each target point and the millimeter-wave radar; Cluster each target point by using the determined position coordinates of each target point to obtain at least one target region; For any target region, divide the target region into grid blocks of a specified size; determine the relative position coordinates of each target point through the position coordinates of the target point with the minimum distance from the millimeter-wave radar in the target region and the position coordinates of each target point; determine the grid block where each target point is located according to the relative position coordinates of each target point and the position of each grid block in the target region; use the preset correspondence between the speed and the gray value of each target point to determine the gray value corresponding to the speed of each target point; set the gray value of the grid block according to the gray value of the target points included in the grid block, and set the gray value of each grid block that does not contain target points to a specified gray value, where the specified gray value is different from the gray value of the grid block containing target points; Input the grid image into a preset target recognition model to determine the target category and the target action corresponding to the target region, wherein the target recognition model is trained in the following manner: Input a target recognition training sample into the target recognition model, extract features of the grid image to obtain target feature information, and match the target feature information with the saved target category features to determine the target category feature with the highest matching degree; the training sample includes a grid image and a labeled target category, and the labeled target category includes a target category and a target action; Compare the target category feature with the highest matching degree with the labeled target category to obtain an error value; When the error value does not meet the specified condition, after adjusting the training parameters of the target recognition model, return to execute the step of inputting the target recognition training samples into the target recognition model until the error value meets the specified condition, and then end the training of the target recognition model.

6. The method according to claim 5, wherein, after clustering each target point by using the determined position coordinates of each target point to obtain at least one target area, the method further includes: determining the position coordinates of the target area according to the position coordinates of each target point in each target area; and, determining the speed of the target area by the speeds of each target point in each target area.

Images