A target detection method, device, electronic device and storage medium

Abstract

The present disclosure provides an object detection method, apparatus, electronic device, and storage medium. The method includes: obtaining multiple frames of point cloud data scanned by a radar device, and time information of each frame of point cloud data scanned; determining position information of a target to be detected based on each frame of point cloud data; determining scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data based on the position information of the target to be detected in each frame of point cloud data; and determining movement information of the target to be detected according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the time information of each frame of point cloud data scanned. The present disclosure determines the movement information of the target by combining the time information of each frame of point cloud data scanned and the relevant information of the target to be detected in each frame of point cloud data, with relatively high accuracy.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to an object detection method, apparatus, electronic device, and storage medium. Background Art

[0002] Currently, in a Motor Vehicle Auto Driving System (MVADS) or an Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), object detection based on lidar has become increasingly important. Among them, lidar emits a laser beam through rotational scanning to form a scanning cross-section, thereby obtaining point cloud data.

[0003] When detecting the movement information of an object, it can be determined based on the scanning timestamp of the object scanned in each frame of point cloud data. In the related art, the timestamp of the point cloud data is usually used as the scanning timestamp of the scanned object. Here, the end time of the point cloud scan can usually be selected as the timestamp of the point cloud data, or the middle moment between the start time and the end time of the point cloud scan can also be selected as the timestamp of the point cloud data.

[0004] However, no matter which of the above methods is used to determine the timestamp of the point cloud data, the time when the object is scanned is not substantially the same as this timestamp. Therefore, if the above object detection scheme is still used to determine the movement information of the object, the detection accuracy will be low. Summary of the Invention

[0005] At least one object detection scheme is provided in the embodiments of the present disclosure. By combining the time information of each frame of point cloud data obtained by scanning and the relevant information of the object to be detected in each frame of point cloud data, the movement information of the object is determined, and the accuracy is relatively high.

[0006] In a first aspect, an object detection method is provided in the embodiments of the present disclosure. The method includes:

[0007] Obtain multiple frames of point cloud data obtained by scanning with a radar device, and the time information of each frame of point cloud data obtained by scanning;

[0008] Based on each frame of point cloud data, determine the position information of the object to be detected;

[0009] Based on the position information of the object to be detected in each frame of point cloud data, determine the scanning direction angle information of the object to be detected scanned by the radar device in each frame of point cloud data;

[0010] Determine the movement information of the target to be detected based on the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned by the radar device, and the time information of each frame of point cloud data obtained by scanning.

[0011] For the above target detection method, based on the position information of the target to be detected in each frame of point cloud data, the moving trajectory points of the target to be detected during the scanning process of the radar device can be determined. Based on the relevant offset information between each moving trajectory point, relatively accurate scanning direction angle information can be determined. Then, combined with the time information of each frame of point cloud data, more accurate movement information (such as movement speed information) of the target to be detected can be obtained.

[0012] In one implementation, the time information of each frame of point cloud data obtained by scanning includes the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data. The determining of the movement information of the target to be detected based on the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned by the radar device, and the time information of each frame of point cloud data obtained by scanning includes:

[0013] Determine the movement information of the target to be detected based on the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned by the radar device, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data.

[0014] Here, the above scanning start and end time information and the scanning start and end angle information are used as scanning calibration information, which is not affected by the misalignment between the scanning time and the time stamp of the point cloud data. In this way, after determining the scanning direction angle corresponding to the target to be detected, the movement information of the target to be detected can be determined with reference to the above scanning calibration information, making the determined movement information more accurate.

[0015] In one implementation, the determining of the movement information of the target to be detected based on the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned by the radar device, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data includes:

[0016] For each frame of point cloud data, based on the scanning direction angle information when the target to be detected in this frame of point cloud data is scanned, and the scanning start and end time information and the scanning start and end angle information corresponding to this frame of point cloud data, determine the scanning time information when the target to be detected in this frame of point cloud data is scanned;

[0017] Determine the displacement information of the target to be detected based on the position information of the target to be detected in multiple frames of point cloud data;

[0018] Determine the moving speed information of the target to be detected based on the scanning time information when the target to be detected in the multiple frames of point cloud data is respectively scanned, and the displacement information of the target to be detected.

[0019] Here, first, based on the scanning direction angle information, the scanning start and end time information, and the scanning start and end angle information, the scanning time information when the target to be detected is scanned by the radar device can be determined, and based on the coordinate information corresponding to the target to be detected in the multiple frames of point cloud data, the displacement information of the target to be detected can be determined. Then, based on the scanning time information and the corresponding movement information, the moving speed information of the target to be detected can be determined.

[0020] In one implementation manner, for each frame of point cloud data, determining the scanning time information when the target to be detected in this frame of point cloud data is scanned based on the scanning direction angle information when the target to be detected in this frame of point cloud data is scanned, and the corresponding scanning start and end time information and scanning start and end angle information of this frame of point cloud data includes:

[0021] For each frame of point cloud data, based on the scanning direction angle information when the target to be detected in this frame of point cloud data is scanned, and the scanning start angle information in the corresponding scanning start and end angle information of this frame of point cloud data, determine the first angle difference between the direction angle of the target to be detected and the scanning start angle; and,

[0022] Based on the scanning end angle information in the corresponding scanning start and end angle information of this frame of point cloud data, and the scanning start angle information, determine the second angle difference between the scanning end angle and the scanning start angle; and,

[0023] Based on the scanning end time information when scanning this frame of point cloud data ends in the corresponding scanning start and end time information of this frame of point cloud data, and the scanning start time information when starting to scan this frame of point cloud data in the corresponding scanning start and end time information of this frame of point cloud data, determine the time difference between the scanning end time information and the scanning start time information;

[0024] Based on the first angle difference, the second angle difference, the time difference, and the scanning start time information, determine the scanning time information when the target to be detected in this frame of point cloud data is scanned.

[0025] To determine the scanning time information when the target to be detected is scanned by the radar device, it is possible to first determine the first angular difference between the direction angle of the target to be detected and the scanning start angle, the second angular difference between the scanning end angle and the scanning start angle, and the time difference between the scanning end time information and the scanning start time information in each frame of point cloud data, and then determine the scanning time information when the target to be detected in this frame of point cloud data is scanned based on the calculation results among the above first angular difference, second angular difference, and time difference.

[0026] In one implementation, the method further includes:

[0027] Controlling the intelligent device based on the moving speed information of the target to be detected and the speed information of the intelligent device provided with the radar device.

[0028] In one implementation, the method further includes:

[0029] Predicting the motion trajectory of the target to be detected in a future time period based on the moving speed information and historical motion trajectory information of the target to be detected.

[0030] In one implementation, determining the position information of the target to be detected based on each frame of point cloud data includes:

[0031] Performing rasterization processing on each frame of point cloud data to obtain a raster matrix; the value of each element in the raster matrix is used to represent whether there is a point cloud point at the corresponding raster.

[0032] Generating a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected.

[0033] Determining the position information of the target to be detected based on the generated sparse matrix.

[0034] In the embodiments of the present disclosure, when each frame of point cloud data is obtained, rasterization processing can be first performed on this frame of point cloud data to obtain a raster matrix, and the value of the element in the raster matrix can represent whether there is a point cloud point at the corresponding raster. In this way, the elements in the raster matrix representing that there is a point cloud point at the corresponding raster can be processed according to the size information of the target to be detected in the target scene to generate a sparse matrix corresponding to the target to be detected, so as to determine the position information of the target to be detected according to the generated sparse matrix.

[0035] In one implementation, generating a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected includes:

[0036] According to the grid matrix and the size information of the target to be detected, perform at least one dilation operation or erosion operation on the target elements in the grid matrix to generate a sparse matrix corresponding to the target to be detected;

[0037] Wherein, the target element is an element representing the existence of point cloud points at the corresponding grid.

[0038] In one implementation manner, the step of performing at least one dilation operation or erosion operation on the target elements in the grid matrix according to the grid matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected includes:

[0039] Perform at least one shift processing and logical operation processing on the target elements in the grid matrix to obtain a sparse matrix corresponding to the target to be detected, wherein the difference between the coordinate range size of the obtained sparse matrix and the size of the target to be detected is within a preset threshold range.

[0040] In one implementation manner, the step of performing at least one dilation operation on the elements in the grid matrix according to the grid matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected includes:

[0041] Perform a first negation operation on the elements in the grid matrix before the current dilation operation to obtain a grid matrix after the first negation operation;

[0042] Perform at least one convolution operation on the grid matrix after the first negation operation based on a first preset convolution kernel to obtain a grid matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected;

[0043] Perform a second negation operation on the elements in the grid matrix with the preset sparsity after at least one convolution operation to obtain the sparse matrix.

[0044] In one implementation manner, the step of performing a first negation operation on the elements in the grid matrix before the current dilation operation to obtain a grid matrix after the first negation operation includes:

[0045] Perform a convolution operation on the elements other than the target elements in the grid matrix before the current dilation operation based on a second preset convolution kernel to obtain first negated elements, and perform a convolution operation on the target elements in the grid matrix before the current dilation operation based on the second preset convolution kernel to obtain second negated elements;

[0046] Based on the first negated elements and the second negated elements, obtain a grid matrix after the first negation operation.

[0047] In one embodiment, performing at least one convolution operation on the raster matrix after the first negation operation based on the first preset convolution kernel to obtain a raster matrix with a preset sparsity after at least one convolution operation includes:

[0048] For the first convolution operation, convolving the raster matrix after the first negation operation with the first preset convolution kernel to obtain a raster matrix after the first convolution operation;

[0049] Determining whether the sparsity of the raster matrix after the first convolution operation reaches the preset sparsity;

[0050] If not, looping to perform the step of convolving the raster matrix after the previous convolution operation with the first preset convolution kernel to obtain a raster matrix after the current convolution operation until a raster matrix with a preset sparsity after at least one convolution operation is obtained.

[0051] Here, for the first convolution operation, the raster matrix after the first convolution operation can be determined based on the convolution operation between the raster matrix after the first negation operation and the first preset convolution kernel, and then the raster matrix after the second convolution operation can be determined based on the raster matrix after the first convolution operation and the first preset convolution kernel, and so on, until a raster matrix with a preset sparsity is obtained.

[0052] In one embodiment, the first convolution kernel has a weight matrix and a bias corresponding to the weight matrix; for the first convolution operation, convolving the raster matrix after the first negation operation with the first preset convolution kernel to obtain a raster matrix after the first convolution operation includes:

[0053] For the first convolution operation, selecting each raster sub-matrix from the raster matrix after the first negation operation according to the size of the first preset convolution kernel and a preset stride;

[0054] For each selected raster sub-matrix, performing a multiplication operation on the raster sub-matrix and the weight matrix to obtain a first operation result, and performing an addition operation on the first operation result and the bias to obtain a second operation result;

[0055] Determining the raster matrix after the first convolution operation based on the second operation results corresponding to the respective raster sub-matrices.

[0056] In one embodiment, performing at least one erosion processing operation on the elements in the raster matrix according to the raster matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected includes:

[0057] Perform at least one convolution operation on the raster matrix to be processed based on a third preset convolution kernel, obtaining a raster matrix with a preset sparsity after the at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected;

[0058] Determine the raster matrix with the preset sparsity after the at least one convolution operation as the sparse matrix corresponding to the target to be detected.

[0059] In one implementation, rasterizing each frame of point cloud data to obtain a raster matrix includes:

[0060] Rasterize each frame of point cloud data to obtain a raster matrix and the corresponding relationship between each element in the raster matrix and the coordinate range information of each point cloud point;

[0061] The determining the position information of the target to be detected based on the generated sparse matrix includes:

[0062] Based on the corresponding relationship between each element in the raster matrix and the coordinate range information of each point cloud point, determine the coordinate information corresponding to each target element in the generated sparse matrix;

[0063] Combine the coordinate information corresponding to each of the target elements in the sparse matrix to determine the position information of the target to be detected.

[0064] Here, based on the corresponding relationship between each element in the raster matrix and the coordinate range information of each point cloud point, the coordinate information of the target elements in the generated sparse matrix can be determined. Then, based on the combination of the coordinate information, the coordinate range of the target to be detected in the sparse matrix can be determined. Subsequently, based on the conversion relationship between the coordinate system where the sparse matrix is located and the physical coordinate system, the position information of the target to be detected can be determined.

[0065] In one implementation, the determining the position information of the target to be detected based on the generated sparse matrix includes:

[0066] Perform at least one convolution process on each target element in the generated sparse matrix based on a trained convolutional neural network to obtain a convolution result;

[0067] Based on the convolution result, determine the position information of the target to be detected.

[0068] Here, the generated sparse matrix can be subjected to convolution processing based on the trained convolutional neural network to determine the position information of the target to be detected through the obtained convolution result. Considering that during the convolution processing, convolution operations can be performed only on the target elements with point cloud points at the corresponding grid positions in the sparse matrix, which reduces the convolution calculation amount to a certain extent and improves the efficiency of target detection.

[0069] In a second aspect, an embodiment of the present disclosure further provides a target detection device, where the device includes:

[0070] An information acquisition module, configured to acquire multiple frames of point cloud data scanned by a radar device, and the time information of each frame of point cloud data scanned;

[0071] A position determination module, configured to determine the position information of the target to be detected based on each frame of point cloud data;

[0072] A direction angle determination module, configured to determine the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data based on the position information of the target to be detected in each frame of point cloud data;

[0073] A target detection module, configured to determine the movement information of the target to be detected according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the time information of each frame of point cloud data scanned.

[0074] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the target detection method described in any one of the first aspect and its various embodiments are executed.

[0075] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the target detection method described in any one of the first aspect and its various embodiments are executed.

[0076] For the effect descriptions of the above target detection device, electronic device, and computer-readable storage medium, refer to the descriptions of the above target detection method, which will not be elaborated here.

[0077] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes the following detailed description. Description of the Drawings

[0078] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required for the embodiments will be briefly introduced below. The accompanying drawings here are incorporated into the specification and form a part of this specification. These accompanying drawings show embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following accompanying drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related accompanying drawings can also be obtained based on these accompanying drawings.

[0079] Figure 1 The flowchart of a target detection method provided in Embodiment 1 of the present disclosure is shown;

[0080] Figure 2 The application schematic diagram of a target detection method provided in Embodiment 1 of the present disclosure is shown;

[0081] Figure 3(a) shows a schematic diagram of a raster matrix before encoding provided in Embodiment 1 of the present disclosure;

[0082] Figure 3(b) shows a schematic diagram of a sparse matrix provided in Embodiment 1 of the present disclosure;

[0083] Figure 3(c) shows a schematic diagram of a raster matrix after encoding provided in Embodiment 1 of the present disclosure;

[0084] Figure 4(a) shows a schematic diagram of a raster matrix after left shift provided in Embodiment 1 of the present disclosure;

[0085] Figure 4(b) shows a schematic diagram of a logical OR operation provided in Embodiment 1 of the present disclosure;

[0086] Figure 5(a) shows a schematic diagram of a raster matrix after the first negation operation provided in Embodiment 1 of the present disclosure;

[0087] Figure 5(b) shows a schematic diagram of a raster matrix after convolution operation provided in Embodiment 1 of the present disclosure;

[0088] Figure 6 The schematic diagram of a target detection device provided in Embodiment 2 of the present disclosure is shown;

[0089] Figure 7 The schematic diagram of an electronic device provided in Embodiment 3 of the present disclosure is shown. Detailed implementation manners

[0090] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only a part rather than all of the embodiments of the present disclosure. The components of the embodiments of the present disclosure usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0091] It has been found through research that when detecting the movement information of a target, it can be determined based on the scanning timestamps of the target scanned in each frame of point cloud data. In related technologies, the timestamp of the point cloud data is usually used as the scanning timestamp of the target scanned. However, based on the imaging principle of lidar, the time when the target is scanned is substantially different from the timestamp of the point cloud data. If the above target detection scheme is still used to determine the movement information of the target, the detection accuracy will be relatively low.

[0092] Based on the above research, the present disclosure provides at least one target detection scheme, which combines the time information of each frame of point cloud data obtained by scanning and the relevant information of the target to be detected in each frame of point cloud data to determine the movement information of the target, with relatively high accuracy.

[0093] All the defects existing in the above solutions are the results obtained by the inventors through practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by the present disclosure below for the above problems should be the contributions made by the inventors to the present disclosure during the process of the present disclosure.

[0094] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0095] To facilitate the understanding of this embodiment, a target detection method disclosed in this disclosure embodiment will be introduced in detail first. The execution subject of the target detection method provided in this disclosure embodiment is generally an electronic device with certain computing capabilities. Such an electronic device includes, for example: a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the target detection method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0096] Next, the target detection method provided in this disclosure embodiment will be described by taking the execution subject as a terminal device as an example.

[0097] Embodiment 1

[0098] See Figure 1 As shown, it is a flowchart of the target detection method provided in this disclosure embodiment. The method includes steps S101 to S104, where:

[0099] S101. Obtain multiple frames of point cloud data scanned by a radar device, and the time information of each frame of point cloud data scanned.

[0100] S102. Based on each frame of point cloud data, determine the position information of the target to be detected.

[0101] S103. Based on the position information of the target to be detected in each frame of point cloud data, determine the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data.

[0102] S104. According to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the time information of each frame of point cloud data scanned, determine the movement information of the target to be detected.

[0103] Here, for the convenience of understanding the object detection method provided by the embodiments of the present disclosure, first, a technical scenario of the object detection method will be briefly described. The object detection method provided by the embodiments of the present disclosure can be applied to a radar device. Taking a rotary scanning radar as an example, when the rotary scanning radar rotates and scans in the horizontal direction, it can obtain point cloud data of targets in the surrounding environment. During the rotary scanning process, the lidar can adopt a multi-line scanning method, that is, multiple laser tubes are used to emit sequentially. The structure is that multiple laser tubes are arranged longitudinally. That is, during the rotary scanning in the horizontal direction, multi-layer scanning in the vertical direction is performed. There is a certain angle between each laser tube, and the vertical emission field of view can be 30°-40°. In this way, for each scanning angle of the lidar device, a data packet returned by the laser emitted by multiple laser tubes can be obtained. By splicing the data packets obtained at each scanning angle, point cloud data can be obtained.

[0104] Based on the above scanning principle of the lidar, the moments when the targets are scanned by the lidar are different. If the time stamp of the point cloud data is directly considered as the common time stamp of all targets, a noise of size T will be introduced to the time stamp of the target, where T is the time consumed for scanning this frame of point cloud, which will result in poor accuracy of the determined moving target.

[0105] It is precisely to solve this problem that the embodiments of the present disclosure provide a solution for determining the movement information of a target by combining the time information of each frame of point cloud data obtained by scanning and the relevant information of the target to be detected in each frame of point cloud data.

[0106] A frame of point cloud data in the embodiments of the present disclosure can be a data set of each point cloud point obtained by splicing multiple data packets scanned in a rotation period (corresponding to a 360° rotation angle), or can be a data set of each point cloud point obtained by splicing data packets scanned in a half rotation period (corresponding to a 180° rotation angle), or can be a data set of each point cloud point obtained by splicing data packets scanned in a quarter rotation period (corresponding to a 90° rotation angle).

[0107] In this way, after determining the position information of the target to be detected based on each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned can be determined based on this position information. Based on this offset angle information and the time information required to scan a frame of point cloud data, the scanning time information when the target to be detected in each frame of point cloud data is scanned can be determined. Then, by combining the position information of the target to be detected in each frame of point cloud data, the movement information of the target to be detected can be determined.

[0108] Among them, the scanning direction angle information corresponding to the target to be detected can indicate the offset angle of the target to be detected from the defined positive X-axis. For example, when the scanning radar starts scanning the target to be detected, at this time, the position of the radar device can be used as the origin, and the direction pointing to the target to be detected can be used as the positive X-axis. At this time, the scanning direction angle of the target to be detected is zero degrees. If the target to be detected is offset by 15° from the positive X-axis, the corresponding scanning direction angle is 15 degrees.

[0109] In a specific application, the corresponding scanning direction angle information can be determined based on the position information of the target to be detected. Here, taking the positive X-axis defined above as the zero-degree direction, based on the cosine relationship of a triangle, the coordinate information can be correspondingly converted into the corresponding scanning direction angle information.

[0110] In the embodiments of the present disclosure, considering that each frame of point cloud data can be collected by selection methods such as a quarter, a half, or a full rotation period, for a frame of point cloud data collected by different selection methods, its scanning start and end angle information will, to a certain extent, affect the scanning time information when the target to be detected in a frame of point cloud data is scanned, and thus affect the determination of the movement information. Therefore, different methods for determining the scanning start and end angle information can be adopted for different selection methods.

[0111] If the selection method adopted in the embodiments of the present disclosure is a full rotation period, the positive X-axis can be used as the scanning start angle, and the scanning end angle corresponding to a full rotation period is 360°. The scanning start and end angle information can be directly determined or determined by using the recording result of the driver of the radar device; if the selection method adopted in the embodiments of the present disclosure is a half or a quarter rotation period, at this time, it is necessary to determine the scanning start and end angle information corresponding to each frame of point cloud data. The scanning start angle and the scanning end angle in the scanning start and end angle information can be the offset angles relative to the positive X-axis, and the scanning start and end angle information can be determined by using the recording result of the driver of the radar device.

[0112] In the embodiments of the present disclosure, when the time information of each frame of point cloud data obtained by scanning includes the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data, the movement information of the target to be detected can be determined based on the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected in each frame of point cloud data is scanned by the radar device, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data.

[0113] Among them, the scanning start and end time information includes the scanning start time information when starting to scan a frame of point cloud data and the scanning end time information when ending the scanning of a frame of point cloud data. The scanning start and end angle information includes the scanning start angle information and the scanning end angle information. The scanning start time information and the scanning start angle information can correspond to the scanning start position when starting to scan a frame of point cloud data, and the scanning end time information and the scanning end angle information can correspond to the scanning end position when ending the scanning of a frame of point cloud data.

[0114] In the case of determining the scanning direction angle information, the scanning start and end time information, and the scanning start and end angle information, that is, based on the scanning start and end information as a reference, it can be determined in which state the movement information of the target to be detected can make the target to be detected in the scanning position where the above scanning direction angle is located, so that the movement information of the target to be detected can be determined.

[0115] The movement information in the embodiments of the present disclosure may be movement speed information. The embodiments of the present disclosure may determine the above movement speed information according to the following steps:

[0116] Step 1: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in the frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to the frame of point cloud data, determine the scanning time information when the target to be detected is scanned in the frame of point cloud data;

[0117] Step 2: Based on the coordinate information of the target to be detected in multiple frames of point cloud data, determine the displacement information of the target to be detected;

[0118] Step 3: Based on the scanning time information when the target to be detected in multiple frames of point cloud data is respectively scanned, and the displacement information of the target to be detected, determine the movement speed information of the target to be detected.

[0119] Here, the object detection method provided in the embodiments of the present disclosure can determine the scanning time information when the target to be detected is scanned by the radar device for each frame of point cloud data. In this way, based on the above scanning time information, the scanning time difference information of the target to be detected corresponding to two frames of point cloud data can be determined. In the case of determining the displacement information of the target to be detected, the displacement information and the above scanning time difference information can be ratio-operated by using a speed calculation method, so as to obtain the movement speed information of the target to be detected. Among them, the movement speed information of the target to be detected includes the speed of movement of the target to be detected and / or the acceleration of movement.

[0120] Among them, in the process of determining the displacement information of the target to be detected, first, based on the position information of each frame of point cloud data of the target to be detected in multiple frames of point cloud data, the position offset of the target to be detected in the point cloud data under two frames of point cloud data can be determined. Mapping this position offset to the actual scene can determine the displacement information of the target to be detected.

[0121] In addition, for each frame of point cloud data, in the process of determining the scanning time information when the target to be detected is scanned in this frame of point cloud data, it can be determined based on the scanning direction angle information when the target to be detected is scanned by this frame of point cloud data, as well as the scanning start and end time information and scanning start and end angle information corresponding to this frame of point cloud data.

[0122] Among them, the above scanning start and end time information and scanning start and end angle information can be recorded by the driver built in the radar device. Theoretically speaking, the radar device has a rated operating frequency, and a common operating frequency is 10 Hertz (HZ). In this way, 10 frames of point cloud data can be output in 1 second. For each frame of point cloud data, the time difference between the scanning end time and the scanning start time can be 100 milliseconds. For a 360° ring-scanning radar device, the start angle and end angle of one frame of point cloud data generally coincide, that is, the angle difference between the scanning end angle and the scanning start angle can be 360°.

[0123] However, during the actual operation of the radar device, due to reasons such as mechanical wear, external resistance, and data loss, the above time difference may be less than 100 milliseconds and the angle difference may be less than 360°. To ensure the accuracy of the scanning time information determined in the embodiments of the present disclosure, the embodiments of the present disclosure use the driver built in the radar device to record the above scanning start and end time information and scanning start and end angle information in real time. That is, what the embodiments of the present disclosure adopt can be actual measurement values. For example, the time difference is 99 milliseconds and the angle difference is 359°.

[0124] In this way, based on the above actual measurement values of the scanning start and end information, and the scanning direction angle information corresponding to the target to be detected, more accurate scanning time information can be obtained. In each frame of point cloud data, the determination process of the scanning time information when the target to be detected is scanned can be achieved through the following steps:

[0125] Step 1: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data and the scanning start angle information in the scanning start and end angle information corresponding to this frame of point cloud data, determine the first angle difference between the direction angle of the target to be detected and the scanning start angle; and, based on the scanning end angle information in the scanning start and end angle information corresponding to this frame of point cloud data and the scanning start angle information, determine the second angle difference between the scanning end angle and the scanning start angle; and, based on the scanning end time information when scanning this frame of point cloud data ends in the scanning start and end time information corresponding to this frame of point cloud data and the scanning start time information when starting to scan this frame of point cloud data in the scanning start and end time information corresponding to this frame of point cloud data, determine the time difference between the scanning end time information and the scanning start time information;

[0126] Step 2: Based on the first angle difference, the second angle difference, the time difference, and the scanning start time information, determine the scanning time information when the target to be detected is scanned in this frame of point cloud data.

[0127] Here, when determining the scanning time information corresponding to the target to be detected, on the premise of determining the scanning start time information, the scanning duration from the scanning start time to when the target to be detected is scanned can be determined. Here, the scanning duration can be determined based on the time difference and the proportion of the angle difference determined by the ratio operation between the first angle difference and the second angle difference. In this way, based on the scanning start time, after calculating the determined scanning duration, the scanning time information of the target to be detected can be obtained.

[0128] Here, considering that the scanning process of the radar device can be uniform, the scanned angle can occupy a certain proportion of a complete circle (corresponding to the angle difference between the scanning end angle and the scanning start angle). When it is determined that there is a target to be detected at a scanning position, this proportional relationship can be used to determine the scanning time information corresponding to the target to be detected.

[0129] To facilitate understanding the determination process of the above scanning time information, it can be combined with Figure 2 for specific illustration.

[0130] As Figure 2 shown, the radar device starts scanning from the scanning start position corresponding to (t 1 , a 1 ), scans clockwise here, continues to scan clockwise after scanning to the target position to be detected corresponding to (t 3 , a 3 ), and stops scanning until it scans to the scanning end position corresponding to (t 2 , a 2 ). Among them, the above t3 , t 2 and t 1 are respectively used to represent the scanning time information, scanning termination time information, and scanning start time information corresponding to the target to be detected; a 3 , a 2 and a 1 are respectively used to represent the scanning direction angle information, scanning termination angle information, and scanning start angle information corresponding to the target to be detected.

[0131] It should be noted that before determining the scanning time information in the object detection method provided by the embodiments of the present disclosure, it is necessary to sense the target to be detected from the point cloud data. For example, for the point cloud data collected in real time, it is possible to find the point cloud block with the highest similarity to the target point cloud based on the point cloud feature description vector inside, and thus determine the target to be detected. Generally, three-dimensional (3-Dimensional) boxes, two-dimensional (2-Dimensional) boxes, polygons and other representation methods can be used. The specific representation method is related to the specific sensing method adopted, and no specific limitation is made here.

[0132] Regardless of which method is used to determine the target, the time when the geometric center of the target is scanned by the laser can be used as the timestamp of the target (corresponding to the scanning time information). Here, the target can be abstracted as a geometric mass point in the lidar coordinate system.

[0133] If the pre-perception algorithm gives a 3D box, the center point of the 3D box can be used as the geometric center. If the pre-perception algorithm gives a 2D box on the top view, the center point of the 2D box can be used as the geometric center (as Figure 2 shown), if the pre-algorithm gives a polygon on the top view, the average coordinates of the polygon nodes can be used as the geometric center. In this way, based on the geometric center of the target, the offset angle of the line connecting the geometric center point and the origin of the lidar coordinate system relative to the positive X-axis can be determined, that is, the scanning direction angle information a 3 corresponding to the target to be detected is determined.

[0134] It can be known from Figure 2 that a 2 - a 1 is less than 360°, that is, the actual measured angle difference is used here. In this way, the angle scanned by the radar device will occupy a certain proportion of the total scanning angle, and can be described by the following equation:

[0135]

[0136] where a 3 - a 1 is used to represent the first angle difference between the direction angle of the target to be detected and the scanning start angle; a2 -a 1 Used to represent the second angular difference t between the scanning end angle and the scanning start angle 2 -t 1 Used to represent the time difference between the scanning end time information and the scanning start time information.

[0137] As can be seen from the above formula, when the radar device scans the target to be detected, the proportion of the scanned angle in the total angle is consistent with the proportion of the elapsed time in the total time. Thus, the above formula is transformed into the following expression regarding t 3 That is:

[0138]

[0139] It can be seen that the target detection method provided by the embodiments of the present disclosure can determine the scanning time information when the target to be detected is scanned based on the first angular difference, the second angular difference, the time difference, and the scanning start time information.

[0140] When determining the scanning time information corresponding to the target to be detected, first, the proportion of the angular difference corresponding to the target to be detected can be determined based on the ratio operation between the first angular difference and the second angular difference. Then, the product operation of the angular difference proportion and the time difference can be performed to obtain the scanning duration from the scanning start time to when the target to be detected is scanned. Finally, by summing the scanning duration and the scanning start time information, the corresponding scanning time information can be obtained.

[0141] After determining the scanning time information according to the above method, the moving speed information of the target to be detected is further determined.

[0142] Considering the key role of the position information of the target to be detected in determining the movement information of the target to be detected, the following will be described in detail.

[0143] In the target detection method provided by the embodiments of the present disclosure, the position information of the target to be detected can be determined according to the following steps:

[0144] Step 1: Perform rasterization processing on each frame of point cloud data to obtain a raster matrix; the value of each element in the raster matrix is used to represent whether there is a point cloud point at the corresponding raster;

[0145] Step 2: Generate a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected;

[0146] Step 3: Determine the position information of the target to be detected based on the generated sparse matrix.

[0147] In the embodiments of the present disclosure, for each frame of point cloud data, rasterization processing can be first performed, and then the raster matrix obtained by the rasterization processing can be sparsified to generate a sparse matrix. The process of rasterization processing here can refer to the process of mapping the point cloud data including each point cloud point distributed in space into a set raster, and performing raster encoding (corresponding to a binary matrix) based on the point cloud points corresponding to the raster. The process of sparsification processing can be a process of performing a dilation operation (corresponding to the processing result of increasing the elements indicated as 1 in the binary matrix) or an erosion operation (corresponding to the processing result of reducing the elements indicated as 1 in the binary matrix) on the above binary matrix based on the size information of the target to be detected in the target scene. Next, the above processes of rasterization processing and sparsification processing will be further described.

[0148] Among them, in the process of the above rasterization processing, the point cloud points distributed in the Cartesian continuous real number coordinate system can be converted to the rasterized discrete coordinate system.

[0149] To facilitate understanding of the above rasterization processing process, a specific example will be described below. The embodiments of the present disclosure have point cloud points such as point A(0.32m, 0.48m), point B(0.6m, 0.4801m), and point C(2.1m, 3.2m). Rasterization is performed with a raster width of 1m. The range from (0m, 0m) to (1m, 1m) corresponds to the first raster, the range from (0m, 1m) to (1m, 2m) corresponds to the second raster, and so on. After rasterization, A′(0, 0) and B′(0, 0) are both in the raster of the first row and the first column, and C′(2, 3) can be in the raster of the second row and the third column, thus realizing the conversion from the Cartesian continuous real number coordinate system to the discrete coordinate system. Among them, the coordinate information of the point cloud points can be determined with reference to a reference point (such as the position of the radar device that collects the point cloud data), which will not be elaborated here.

[0150] In the embodiments of the present disclosure, two-dimensional rasterization can be performed, or three-dimensional rasterization can be performed. Three-dimensional rasterization adds height information on the basis of two-dimensional rasterization. Next, two-dimensional rasterization will be specifically described as an example.

[0151] For two-dimensional rasterization, a finite space can be divided into N*M rasters, generally equally spaced, and the interval size can be configured. At this time, the rasterized point cloud data can be encoded using a binary matrix (i.e., the above raster matrix). Each raster can be represented by a coordinate composed of a unique row number and column number. If there is one or more point cloud points in the raster, the raster is encoded as 1, otherwise as 0, so that the encoded binary matrix can be obtained.

[0152] After determining the grid matrix according to the above method, the elements in the above grid matrix can be sparsely processed according to the size information of the target to be detected, so as to generate a corresponding sparse matrix.

[0153] Among them, the size information of the target to be detected can be obtained in advance. Here, the size information of the target to be detected can be determined by combining the image data synchronously collected with the point cloud data, or can be roughly estimated based on the specific application scenario of the object detection method provided in this embodiment of the present disclosure. For example, in the field of autonomous driving, the object in front of the vehicle can be the vehicle, and its general size information can be determined to be 4m×4m. In addition, the size information of the target to be detected can also be determined based on other methods in this embodiment of the present disclosure, and this embodiment of the present disclosure does not make specific limitations on this.

[0154] In this embodiment of the present disclosure, the sparse processing operation can be to perform at least one dilation processing operation on the target elements in the grid matrix (that is, the elements representing the existence of point cloud points at the corresponding grid). Here, the dilation processing operation can be performed when the size of the coordinate range of the grid matrix is smaller than the size of the target to be detected in the target scene. That is, through one or more dilation processing operations, the range of elements representing the existence of point cloud points at the corresponding grid can be gradually expanded, so that the expanded element range can match the target to be detected, and thus the position determination can be realized. In addition, the sparse processing operation in this embodiment of the present disclosure can also be to perform at least one erosion processing operation on the target elements in the grid matrix. Here, the erosion processing operation can be performed when the size of the coordinate range of the grid matrix is larger than the size of the target to be detected in the target scene. That is, through one or more erosion processing operations, the range of elements representing the existence of point cloud points at the corresponding grid can be gradually reduced, so that the reduced element range can match the target to be detected, and thus the position determination can be realized.

[0155] In specific applications, whether to perform one dilation processing operation, multiple dilation processing operations, one erosion processing operation, or multiple erosion processing operations depends on whether the difference between the size of the coordinate range of the sparse matrix obtained by performing at least one shift processing and logical operation processing and the size of the target to be detected in the target scene is within a preset threshold range. That is, the dilation or erosion processing operation adopted in the present disclosure is based on the constraint of the size information of the target to be detected, so that the information represented by the determined sparse matrix is more in line with the relevant information of the target to be detected.

[0156] It can be understood that the purpose of the sparse processing implemented by either the dilation processing operation or the erosion processing operation is to make the generated sparse matrix be able to represent more accurate relevant information of the target to be detected.

[0157] In the embodiments of the present disclosure, the above dilation processing operation may be implemented based on a shift operation and a logical OR operation, or may be implemented based on inversion after convolution and then inversion after convolution. Although the specific methods adopted by the two operations are different, the effects of the finally generated sparse matrices may be the same.

[0158] In addition, the above erosion processing operation may be implemented based on a shift operation and a logical AND operation, or may be directly implemented based on a convolution operation. Similarly, although the specific methods adopted by the two operations are different, the effects of the finally generated sparse matrices may also be the same.

[0159] Next, taking the dilation processing operation as an example, combined with Figures 3(a) to 3(b) the specific example diagram of generating a sparse matrix shown in the figure, the generation process of the above sparse matrix will be further described.

[0160] As shown in FIG. 3(a), it is a schematic diagram of the raster matrix obtained after rasterization processing (corresponding to before encoding). By performing a dilation operation on each target element (corresponding to the raster with a filling effect) in the raster matrix, the corresponding sparse matrix shown in FIG. 3(b) can be obtained. It can be known that in the embodiments of the present disclosure, for the target elements with point cloud points at the corresponding raster positions in FIG. 3(a), a dilation operation of eight neighborhoods is performed, so that each target element becomes an element set after dilation, and the raster width corresponding to the element set can match the size of the target to be detected.

[0161] Among them, the above dilation operation of eight neighborhoods may be a process of determining elements whose absolute values of the differences in the abscissa or ordinate from this element do not exceed 1. Except for the elements at the raster edge, generally there are eight elements in the neighborhood of an element (corresponding to the above element set). The input of the dilation processing result may be the coordinate information of 6 target elements, and the output may be the coordinate information of the element set within the eight neighborhoods of the target element, as shown in FIG. 3(b).

[0162] It should be noted that in practical applications, in addition to the above dilation operation of eight neighborhoods, a dilation operation of four neighborhoods or other dilation operations may also be performed, which is not specifically limited herein. In addition, the embodiments of the present disclosure may also perform multiple dilation operations. For example, based on the dilation result shown in FIG. 3(b), a dilation operation is performed again to obtain a sparse matrix with a larger element set range, which will not be elaborated herein.

[0163] In the embodiments of the present disclosure, based on the generated sparse matrix, the position information of the target to be detected can be determined. In the embodiments of the present disclosure, it can be specifically implemented through the following two aspects.

[0164] First aspect: Here, the position information of the target to be detected can be determined based on the correspondence between each element in the grid matrix and the coordinate range information of each point cloud point. Specifically, it can be implemented through the following steps:

[0165] Step 1: Based on the correspondence between each element in the grid matrix and the coordinate range information of each point cloud point, determine the coordinate information corresponding to each target element in the generated sparse matrix;

[0166] Step 2: Combine the coordinate information corresponding to each target element in the sparse matrix to determine the position information of the target to be detected.

[0167] Here, based on the above relevant descriptions of rasterization processing, it can be known that each target element in the grid matrix can correspond to multiple point cloud points. In this way, the point cloud point coordinate range information corresponding to the element and multiple point cloud points can be determined in advance. Here, still taking the grid matrix of N*M dimension as an example, the target element with point cloud points can correspond to P point cloud points, and the coordinate of each point is (Xi, Yi), where i belongs to 0 to P-1, and Xi, Yi represent the position of the point cloud point in the grid matrix, 0 <= Xi < N, 0 <= Yi < M.

[0168] In this way, after generating the sparse matrix, the correspondence between each of the above-mentioned elements and the coordinate range information of each point cloud point determined in advance can be used to determine the coordinate information corresponding to each target element in the sparse matrix, that is, the inverse rasterization processing operation is performed.

[0169] It should be noted that since the sparse matrix is obtained by performing sparse processing on the elements in the grid matrix that represent the existence of point cloud points at the corresponding grids, therefore, the target elements in the sparse matrix here can represent the elements where there are point cloud points at the corresponding grids.

[0170] To facilitate the understanding of the above inverse rasterization processing process, a specific example will be given below for illustration. Here, taking the point A′(0, 0) and point B′(0, 0) indicated by the sparse matrix in the first row and first column grid, and point C′(2, 3) in the second row and third column grid as an example, during the inverse rasterization processing, for the first grid (0, 0), after mapping its center back to the Cartesian coordinate system, (0.5m, 0.5m) can be obtained. For the grid (2, 3) in the second row and third column, after mapping its center back to the Cartesian coordinate system, (2.5m, 3.5m) can be obtained. That is, (0.5m, 0.5m) and (2.5m, 3.5m) can be determined as the mapped coordinate information. In this way, by combining the mapped coordinate information, the position information of the target to be detected can be determined.

[0171] The embodiments of the present disclosure can not only determine the position information of the target to be detected based on the approximate relationship between the sparse matrix and the target detection result, but also determine the position information of the target to be detected based on the trained convolutional neural network.

[0172] Second aspect: The embodiments of the present disclosure can first perform at least one convolution process on the generated sparse matrix based on the trained convolutional neural network, and then can determine the position information of the target to be detected based on the convolution result obtained from the convolution process.

[0173] In the related technologies that use convolutional neural networks to implement target detection, it is necessary to traverse all the input data, sequentially find the neighborhood points of the input points for convolution operations, and finally output the set of all neighborhood points. However, the target detection method provided by the embodiments of the present disclosure only needs to quickly traverse the target elements in the sparse matrix to find the positions of the valid points (i.e., the elements with a value of 1 in the binary matrix) for convolution operations, thereby greatly accelerating the calculation process of the convolutional neural network and improving the efficiency of determining the position information of the target to be detected.

[0174] Considering the key role of the sparse processing operation in the target detection method provided by the embodiments of the present disclosure, the following can be described separately from two aspects.

[0175] First aspect: In the case where the sparse processing operation is a dilation processing operation, the embodiments of the present disclosure can be implemented by combining shift processing and logical operations, and can also be implemented based on convolution after inversion and then inversion after convolution.

[0176] Firstly, in the embodiments of the present disclosure, one or more dilation processing operations can be performed based on at least one shift processing and logical OR operation. In the specific implementation process, the number of specific dilation processing operations can be determined in combination with the size information of the target to be detected in the target scenario.

[0177] Here, for the first dilation processing operation, the target elements representing the presence of point cloud points at the corresponding grids can be shifted in multiple preset directions to obtain multiple corresponding shifted grid matrices. Then, a logical OR operation can be performed on the grid matrix and the multiple shifted grid matrices corresponding to the first dilation processing operation, so as to obtain the sparse matrix after the first dilation processing operation. Here, it can be determined whether the size of the coordinate range of the obtained sparse matrix is smaller than the size of the target to be detected, and whether the corresponding difference is large enough (e.g., greater than a preset threshold). If so, the target elements in the sparse matrix after the first dilation processing operation can be shifted in multiple preset directions and a logical OR operation can be performed according to the above method to obtain the sparse matrix after the second dilation processing operation, and so on, until it is determined that the difference between the size of the coordinate range of the latest obtained sparse matrix and the size of the target to be detected in the target scene belongs to the preset threshold range, and then the sparse matrix is determined.

[0178] It should be noted that, regardless of which dilation processing operation the obtained sparse matrix is after, it is essentially a binary matrix. As the number of dilation processing operations increases, the number of target elements representing the presence of point cloud points at the corresponding grids in the obtained sparse matrix also increases. And since the grids mapped by the binary matrix have width information, here, the size of the coordinate range corresponding to each target element in the sparse matrix can be used to verify whether the size of the target to be detected in the target scene is reached, thereby improving the accuracy of subsequent target detection applications.

[0179] Among them, the above logical OR operation can be implemented according to the following steps:

[0180] Step 1: Select a shifted grid matrix from the multiple shifted grid matrices;

[0181] Step 2: Perform a logical OR operation on the grid matrix before the current dilation processing operation and the selected shifted grid matrix to obtain an operation result;

[0182] Step 3: Loop to select the unprocessed grid matrices from the multiple shifted grid matrices, and perform a logical OR operation on the selected grid matrix and the result of the most recent operation until all the grid matrices are selected, so as to obtain the sparse matrix after the current dilation processing operation.

[0183] Here, first, one of the shifted grid matrices can be selected from multiple shifted grid matrices. In this way, the grid matrix before the current dilation processing operation can be logically ORed with the selected shifted grid matrix to obtain an operation result. Here, the unprocessed shifted grid matrices can be cyclically selected from multiple shifted grid matrices and participate in the logical OR operation until all the shifted grid matrices are selected, and then the sparse matrix after the current dilation processing operation can be obtained.

[0184] The dilation processing operation in the embodiments of the present disclosure can be four-neighborhood dilation centered on the target element, or eight-neighborhood dilation centered on the target element, or other neighborhood processing operation methods. In specific applications, the corresponding neighborhood processing operation method can be selected based on the size information of the target to be detected, and no specific limitation is made here.

[0185] It should be noted that for different neighborhood processing operation methods, the preset directions of the corresponding shift processing are different. Taking four-neighborhood dilation as an example, the grid matrix can be shifted in four preset directions, namely left shift, right shift, up shift, and down shift. Taking eight-neighborhood dilation as an example, the grid matrix can be shifted in four preset directions, namely left shift, right shift, up shift, down shift, up shift and down shift under the premise of left shift, and up shift and down shift under the premise of right shift. In addition, in order to adapt to the subsequent logical OR operation, after determining the shifted grid matrix based on multiple shift directions, a logical OR operation can be performed first, and then the result of the logical OR operation is shifted in multiple shift directions, and then the next logical OR operation is performed, and so on, until the sparse matrix after the dilation processing is obtained.

[0186] To facilitate the understanding of the above dilation processing operation, the grid matrix before encoding shown in FIG. 3(a) can be first converted into the grid matrix after encoding shown in FIG. 3(c), and then an example of the first dilation processing operation is described in combination with Figures 4(a) to 4(b) the first dilation processing operation.

[0187] As shown in the grid matrix of FIG. 3(c), as a binary matrix, the positions of all 1s in the matrix can represent the grids where the target elements are located, and all 0s in the matrix can represent the background.

[0188] In the embodiments of the present disclosure, first, matrix shifting can be used to determine the neighborhoods of the elements with a value of 1 in the binary matrix. Here, shifting processes in four preset directions can be defined, namely left shift, right shift, up shift, and down shift. Among them, for the left shift, the column coordinates corresponding to the elements with a value of 1 in the binary matrix are decreased by 1, as shown in FIG. 4(a); for the right shift, the column coordinates corresponding to the elements with a value of 1 in the binary matrix are increased by 1; for the up shift, the row coordinates corresponding to the elements with a value of 1 in the binary matrix are decreased by 1; for the down shift, the row coordinates corresponding to the elements with a value of 1 in the binary matrix are increased by 1.

[0189] Secondly, in the embodiments of the present disclosure, matrix logical OR operations can be used to merge the results of all neighborhoods. Matrix logical OR means that when two binary matrices of the same size are received as inputs, logical OR operations are sequentially performed on the binary values at the same positions of the two matrices, and the resulting values form a new binary matrix as the output. As shown in FIG. 4(b), a specific example of a logical OR operation is given.

[0190] In the specific process of implementing the logical OR operation, the raster matrices after left shift, right shift, up shift, and down shift can be sequentially selected to participate in the logical OR operation. For example, the raster matrix can be logically ORed with the raster matrix after left shift first, and the resulting operation result can then be logically ORed with the raster matrix after right shift. For the resulting operation result, it can be logically ORed with the raster matrix after up shift, and for the resulting operation result, it can be logically ORed with the raster matrix after down shift, so as to obtain the sparse matrix after the first dilation processing operation.

[0191] It should be noted that the above selection order of the translated raster matrices is only a specific example. In practical applications, other methods can also be combined for selection. Considering the symmetry of the translation operation, here, the up shift and down shift can be paired for logical OR operations, and the left shift and right shift can be paired for logical operations. The two logical OR operations can be performed synchronously, which can save calculation time.

[0192] Second, in the embodiments of the present disclosure, dilation processing operations can be implemented by combining convolution and two negation processes. Specifically, it can be implemented through the following steps:

[0193] Step 1: Perform a first negation operation on the elements in the raster matrix before the current dilation processing operation to obtain the raster matrix after the first negation operation;

[0194] Step 2: Perform at least one convolution operation on the raster matrix after the first negation operation based on a first preset convolution kernel to obtain a raster matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected in the target scenario;

[0195] Step 3: Perform a second negation operation on the elements in the raster matrix with a preset sparsity after at least one convolution operation to obtain a sparse matrix.

[0196] In the embodiments of the present disclosure, the dilation processing operation can be implemented through the operations of negating after convolution and then negating after convolution. To a certain extent, the obtained sparse matrix can also represent the relevant information of the target to be detected. In addition, considering that the above convolution operation can be automatically combined with the convolutional neural network used in subsequent applications such as target detection, the detection efficiency can be improved to a certain extent.

[0197] In the embodiments of the present disclosure, the negation operation can be implemented based on the convolution operation or other negation operation methods. For the convenience of cooperating with the subsequent application network (such as the convolutional neural network used for target detection), here, the convolution operation can be specifically used to implement it. Next, the above first negation operation will be specifically described.

[0198] Here, a first negated element can be obtained by performing a convolution operation on the elements other than the target element in the raster matrix before the current dilation processing operation based on a second preset convolution kernel, and a second negated element can also be obtained by performing a convolution operation on the target element in the raster matrix before the current dilation processing operation based on the second preset convolution kernel. Based on the above first negated element and second negated element, the raster matrix after the first negation operation can be determined.

[0199] The implementation process of the second negation operation can refer to the implementation process of the above first negation operation and will not be elaborated here.

[0200] In the embodiments of the present disclosure, at least one convolution operation can be performed on the raster matrix after the first negation operation using a first preset convolution kernel to obtain a raster matrix with a preset sparsity. If the dilation processing operation can be regarded as a means to increase the number of target elements in the raster matrix, then the above convolution operation can be regarded as a process of reducing the number of target elements in the raster matrix (corresponding to the erosion processing operation). Since the convolution operation in the embodiments of the present disclosure is performed on the raster matrix after the first negation operation, the equivalent operation equivalent to the above dilation processing operation can be achieved by using the negation operation in combination with the erosion processing operation and then performing the negation operation again.

[0201] Among them, for the first convolution operation, the raster matrix after the first negation operation is convolved with the first preset convolution kernel to obtain the raster matrix after the first convolution operation. After determining that the sparsity of the raster matrix after the first convolution operation does not reach the preset sparsity, the raster matrix after the first convolution operation can be convolved with the first preset convolution kernel again to obtain the raster matrix after the second convolution operation, and so on, until a raster matrix with a preset sparsity is determined.

[0202] Among them, the above sparsity can be determined by the proportion distribution of target elements and non-target elements in the grid matrix. The more the proportion of target elements, the larger the size information of the target to be detected represented by it. On the contrary, the less the proportion of target elements, the smaller the size information of the target to be detected represented by it. In the embodiments of the present disclosure, the convolution operation can be stopped when the proportion distribution reaches the preset sparsity.

[0203] The convolution operation in the embodiments of the present disclosure can be performed once or multiple times. Here, the specific operation process of the first convolution operation can be used for illustration, including the following steps:

[0204] Step 1: For the first convolution operation, select each grid sub-matrix from the grid matrix after the first negation operation according to the size of the first preset convolution kernel and the preset stride.

[0205] Step 2: For each selected grid sub-matrix, perform a multiplication operation on the grid sub-matrix and the weight matrix to obtain a first operation result, and perform an addition operation on the first operation result and the bias to obtain a second operation result.

[0206] Step 3: Based on the second operation results corresponding to each grid sub-matrix, determine the grid matrix after the first convolution operation.

[0207] Here, a traversal method can be used to traverse the grid matrix after the first negation operation. In this way, for each grid sub-matrix traversed, the grid sub-matrix can be multiplied by the weight matrix to obtain a first operation result, and the first operation result can be added to the bias to obtain a second operation result. In this way, by combining the second operation results corresponding to each grid sub-matrix into the corresponding matrix elements, the grid matrix after the first convolution operation can be obtained.

[0208] To facilitate the understanding of the above dilation processing operation, here, still taking the encoded grid matrix shown in Figure 3(c) as an example, in combination with Figures 5(a) to 5(b) An example of the dilation processing operation is described.

[0209] Here, a 1*1 convolution kernel (i.e., the second preset convolution kernel) can be used to implement the first negation operation. The weight of the second preset convolution kernel is -1 and the bias is 1. At this time, substituting the weight and the bias into the convolution formula {output = input grid matrix * weight + bias}, if the input is a target element in the grid matrix and its value corresponds to 1, then the output = 1 * -1 + 1 = 0; if the input is a non-target element in the grid matrix and its value corresponds to 0, then the output = 0 * -1 + 1 = 1; in this way, after the 1*1 convolution kernel acts on the input, the binary matrix can be negated, and the element value 0 becomes 1 and the element value 1 becomes 0, as shown in Figure 5(a).

[0210] For the above corrosion treatment operation, in specific applications, a 3×3 convolution kernel (i.e., the first preset convolution kernel) and a rectified linear unit (ReLU) can be used to implement it. Each weight included in the weight matrix of the above first preset convolution kernel is 1, and the bias is 8. In this way, the above corrosion treatment operation can be implemented using the formula {output = ReLU (the raster matrix after the first inversion operation of the input * weight + bias)}.

[0211] Here, only when all elements in the input 3×3 raster sub-matrix are 1, the output = ReLU(9 - 8) = 1; otherwise, the output = ReLU (the input raster sub-matrix * 1 - 8) = 0, where (the input raster sub-matrix * 1 - 8) < 0. As shown in Figure 5(b), it is the raster matrix after the convolution operation.

[0212] Here, each time a convolution network with a second preset convolution kernel is nested, a corrosion operation can be superimposed once, so that a raster matrix with a fixed sparsity can be obtained. Taking the inversion operation again can be equivalent to a dilation processing operation, thereby realizing the generation of a sparse matrix.

[0213] Second aspect: When the sparse processing operation is a corrosion treatment operation, the embodiments of the present disclosure can be implemented by combining shift processing and logical operations, and can also be implemented based on convolution operations.

[0214] First, in the embodiments of the present disclosure, one or more corrosion treatment operations can be performed based on at least one shift processing and logical AND operation. In the specific implementation process, the number of specific corrosion treatment operations can be determined in combination with the size information of the target to be detected in the target scenario.

[0215] Similar to the dilation processing implemented based on shift processing and logical OR operation in the first aspect, during the corrosion treatment operation, the shift processing of the raster matrix can also be performed first. Different from the above dilation processing, the logical operation here can be a logical AND operation on the shifted raster matrix. For the process of implementing the corrosion treatment operation based on shift processing and logical AND operation, please refer to the above description for details and will not be elaborated here.

[0216] Similarly, the corrosion treatment operation in the embodiments of the present disclosure can be a four-neighborhood corrosion centered on the target element, an eight-neighborhood corrosion centered on the target element, or other neighborhood processing operation methods. In specific applications, the corresponding neighborhood processing operation method can be selected based on the size information of the target to be detected, and no specific limitation is made here.

[0217] Second, in the embodiments of the present disclosure, erosion processing operations can be implemented in combination with convolution processing, which can be specifically achieved through the following steps:

[0218] Step 1: Perform at least one convolution operation on the grid matrix based on a third preset convolution kernel to obtain a grid matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected in the target scenario;

[0219] Step 2: Determine the grid matrix with a preset sparsity after at least one convolution operation as the sparse matrix corresponding to the target to be detected.

[0220] The above convolution operation can be regarded as a process of reducing the number of target elements in the grid matrix, that is, the erosion processing process. Among them, for the first convolution operation, the grid matrix is convolved with the first preset convolution kernel to obtain the grid matrix after the first convolution operation. After determining that the sparsity of the grid matrix after the first convolution operation does not reach the preset sparsity, the grid matrix after the first convolution operation can be convolved with the third preset convolution kernel again to obtain the grid matrix after the second convolution operation, and so on, until a grid matrix with a preset sparsity can be determined, that is, the sparse matrix corresponding to the target to be detected is obtained.

[0221] The convolution operation in the embodiments of the present disclosure can be one time or multiple times. For the specific process of the convolution operation, refer to the relevant description of implementing dilation processing based on convolution and inversion in the first aspect above, and details will not be repeated here.

[0222] It should be noted that in specific applications, convolution neural networks with different data processing bit widths can be used to implement the generation of sparse matrices. For example, 4 bits (bit) can be used to represent the input, output, and parameters used in the calculation of the network, such as the element values (0 or 1) of the grid matrix, weights, bias amounts, etc. In addition, 8 bits can also be used for representation to adapt to the network processing bit width and improve the operation efficiency.

[0223] In the specific application of the target detection method provided in the embodiments of the present disclosure, the radar device can be set on intelligent devices such as intelligent vehicles, intelligent lamp posts, and robots. In the case where the same target is detected in two adjacent frames of point cloud data scanned by the radar device, if there is a relative displacement of L, the time when the target appears in the first frame is t1, and the time when it appears in the second frame is t2. In the related art, t2 - t1 is equal to the fixed interval T between the two frames, so the speed of the target is L / T. However, t2 - t1 determined by using the above method provided in the embodiments of the present disclosure reflects the time interval when the real target is scanned, which ranges from [0, 2T]. The target speed determined by using this real scanning time interval is also more accurate.

[0224] Based on the above speed determination formula, it can be known that the greater the speed, the greater the corresponding relative displacement. If the speed of the target cannot be accurately judged, it may cause the intelligent device to not be able to well handle the changes brought by the relative displacement. The embodiments of the present disclosure provide a method for accurately determining the scanning time information of the target precisely to solve such problems. In this way, more accurate speed estimation can be achieved, and then combined with the speed information of the intelligent device itself to control the intelligent device to make more reasonable judgments, such as whether to brake suddenly, whether to overtake, etc.

[0225] In the multi-target tracking algorithm, each detection target in the current frame point cloud data can be matched with all the trajectories in the historical frame to obtain the matching similarity, so as to determine which trajectory that the detection target belongs to that has appeared in history. When matching, since the target may be moving, motion compensation can be performed on the historical trajectory. The compensation method can be based on the position and speed of the target in the historical trajectory, and then the position of a target in the current frame can be predicted. Here, an accurate timestamp will make the determined speed more accurate, and further make the predicted position of the target in the current frame more accurate. In this way, even if multi-target tracking is performed, tracking based on the accurate predicted position will greatly reduce the failure rate of target tracking.

[0226] In addition, the target detection method provided by the embodiments of the present disclosure can also predict the motion trajectory of the target to be detected in the future time period based on the moving speed information and historical motion trajectory information of the target to be detected. In specific applications, machine learning methods or other trajectory prediction methods can be used to achieve trajectory prediction. For example, the moving speed information and historical motion trajectory information of the target to be detected can be input into a trained neural network to obtain the predicted motion trajectory in the future time period.

[0227] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation to the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0228] Based on the same inventive concept, the embodiments of the present disclosure also provide a target detection device corresponding to the target detection method. Since the principle of solving problems by the device in the embodiments of the present disclosure is similar to the above target detection method of the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0229] Embodiment 2

[0230] Refer to Figure 6As shown in the figure, it is a schematic structural diagram of an object detection device provided by an embodiment of the present disclosure. The above device includes: an information acquisition module 601, a position determination module 602, a direction angle determination module 603, and an object detection module 604; wherein,

[0231] The information acquisition module 601 is configured to acquire multiple frames of point cloud data scanned by a radar device, and the time information of each frame of point cloud data scanned.

[0232] The position determination module 602 is configured to determine the position information of the object to be detected based on each frame of point cloud data.

[0233] The direction angle determination module 603 is configured to determine the scanning direction angle information when the object to be detected is scanned by the radar device in each frame of point cloud data based on the position information of the object to be detected in each frame of point cloud data.

[0234] The object detection module 604 is configured to determine the movement information of the object to be detected according to the position information of the object to be detected in each frame of point cloud data, the scanning direction angle information when the object to be detected is scanned by the radar device in each frame of point cloud data, and the time information of each frame of point cloud data scanned.

[0235] In one implementation, the object detection module 604 is configured to determine the movement information of the object to be detected according to the following steps:

[0236] Determine the movement information of the object to be detected according to the position information of the object to be detected in each frame of point cloud data, the scanning direction angle information when the object to be detected is scanned by the radar device in each frame of point cloud data, and the scanning start and end time information and scanning start and end angle information corresponding to each frame of point cloud data.

[0237] In one implementation, the object detection module 604 is configured to determine the movement information of the object to be detected according to the following steps:

[0238] For each frame of point cloud data, determine the scanning time information when the object to be detected in this frame of point cloud data is scanned based on the scanning direction angle information when the object to be detected in this frame of point cloud data is scanned, and the scanning start and end time information and scanning start and end angle information corresponding to this frame of point cloud data.

[0239] Determine the displacement information of the object to be detected based on the position information of the object to be detected in multiple frames of point cloud data.

[0240] Determine the movement speed information of the object to be detected based on the scanning time information when the object to be detected in multiple frames of point cloud data is respectively scanned, and the displacement information of the object to be detected.

[0241] In one embodiment, the target detection module 604 is configured to determine the scanning time information when the target to be detected in the frame of point cloud data is scanned according to the following steps:

[0242] For each frame of point cloud data, based on the scanning direction angle information when the target to be detected in the frame of point cloud data is scanned and the scanning start angle information in the scanning start and end angle information corresponding to the frame of point cloud data, determine the first angle difference between the direction angle of the target to be detected and the scanning start angle; and,

[0243] Based on the scanning end angle information and the scanning start angle information in the scanning start and end angle information corresponding to the frame of point cloud data, determine the second angle difference between the scanning end angle and the scanning start angle; and,

[0244] Based on the scanning end time information when scanning the frame of point cloud data ends and the scanning start time information when starting to scan the frame of point cloud data in the scanning start and end time information corresponding to the frame of point cloud data, determine the time difference between the scanning end time information and the scanning start time information;

[0245] Based on the first angle difference, the second angle difference, the time difference, and the scanning start time information, determine the scanning time information when the target to be detected in the frame of point cloud data is scanned.

[0246] In one embodiment, the device further includes:

[0247] The device control module 605 is configured to control the intelligent device based on the moving speed information of the target to be detected and the speed information of the intelligent device provided with the radar device.

[0248] In one embodiment, the above device further includes:

[0249] The trajectory prediction module 606 is configured to predict the motion trajectory of the target to be detected in a future time period based on the moving speed information and the historical motion trajectory information of the target to be detected.

[0250] In one embodiment, the position determination module 602 is configured to determine the position information of the target to be detected based on each frame of point cloud data according to the following steps:

[0251] Perform rasterization processing on each frame of point cloud data to obtain a raster matrix; the value of each element in the raster matrix is used to represent whether there is a point cloud point at the corresponding raster;

[0252] Generate a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected;

[0253] Based on the generated sparse matrix, determine the position information of the target to be detected.

[0254] In one embodiment, the position determination module 602 is configured to generate a sparse matrix corresponding to the target to be detected according to the following steps based on the grid matrix and the size information of the target to be detected:

[0255] Perform at least one dilation processing operation or erosion processing operation on the target elements in the grid matrix according to the grid matrix and the size information of the target to be detected, and generate a sparse matrix corresponding to the target to be detected;

[0256] Wherein, the target element is an element representing that there is a point cloud point at the corresponding grid.

[0257] In one embodiment, the position determination module 602 is configured to generate a sparse matrix corresponding to the target to be detected according to the following steps:

[0258] Perform at least one shift processing and logical operation processing on the target elements in the grid matrix to obtain a sparse matrix corresponding to the target to be detected, wherein the difference between the coordinate range size of the obtained sparse matrix and the size of the target to be detected is within a preset threshold range.

[0259] In one embodiment, the position determination module 602 is configured to generate a sparse matrix corresponding to the target to be detected according to the following steps:

[0260] Perform a first inversion operation on the elements in the grid matrix before the current dilation processing operation to obtain a grid matrix after the first inversion operation;

[0261] Perform at least one convolution operation on the grid matrix after the first inversion operation based on a first preset convolution kernel to obtain a grid matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected;

[0262] Perform a second inversion operation on the elements in the grid matrix with the preset sparsity after at least one convolution operation to obtain a sparse matrix.

[0263] In one embodiment, the position determination module 602 is configured to perform a first inversion operation on the elements in the grid matrix before the current dilation processing operation according to the following steps to obtain a grid matrix after the first inversion operation:

[0264] Perform a convolution operation on other elements except the target elements in the grid matrix before the current dilation processing operation based on a second preset convolution kernel to obtain a first inverted element, and perform a convolution operation on the target elements in the grid matrix before the current dilation processing operation based on a second preset convolution kernel to obtain a second inverted element;

[0265] Based on the first negated element and the second negated element, a raster matrix after the first negation operation is obtained.

[0266] In one embodiment, the position determination module 602 is configured to perform at least one convolution operation on the raster matrix after the first negation operation based on a first preset convolution kernel according to the following steps to obtain a raster matrix with a preset sparsity after at least one convolution operation:

[0267] For the first convolution operation, perform a convolution operation on the raster matrix after the first negation operation and the first preset convolution kernel to obtain a raster matrix after the first convolution operation;

[0268] Determine whether the sparsity of the raster matrix after the first convolution operation reaches the preset sparsity;

[0269] If not, loop to perform the step of performing a convolution operation on the raster matrix after the previous convolution operation and the first preset convolution kernel to obtain a raster matrix after the current convolution operation until a raster matrix with a preset sparsity after at least one convolution operation is obtained.

[0270] In one embodiment, the first convolution kernel has a weight matrix and a bias corresponding to the weight matrix; the position determination module 602 is configured to perform a convolution operation on the raster matrix after the first negation operation and the first preset convolution kernel according to the following steps for the first convolution operation to obtain a raster matrix after the first convolution operation:

[0271] For the first convolution operation, select each raster sub-matrix from the raster matrix after the first negation operation according to the size of the first preset convolution kernel and a preset stride;

[0272] For each selected raster sub-matrix, perform a multiplication operation on the raster sub-matrix and the weight matrix to obtain a first operation result, and perform an addition operation on the first operation result and the bias to obtain a second operation result;

[0273] Based on the second operation results corresponding to each raster sub-matrix, determine the raster matrix after the first convolution operation.

[0274] In one embodiment, the position determination module 602 is configured to perform at least one erosion processing operation on the elements in the raster matrix according to the following steps based on the raster matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected:

[0275] Perform at least one convolution operation on the raster matrix to be processed based on a third preset convolution kernel to obtain a raster matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected;

[0276] Determine the raster matrix with a preset sparsity after at least one convolution operation as the sparse matrix corresponding to the target to be detected.

[0277] In one embodiment, the position determination module 602 is configured to determine the position information of the target to be detected based on the generated sparse matrix according to the following steps:

[0278] Perform rasterization processing on each frame of point cloud data to obtain a raster matrix and the correspondence between each element in the raster matrix and the coordinate range information of each point cloud point;

[0279] Based on the correspondence between each element in the raster matrix and the coordinate range information of each point cloud point, determine the coordinate information corresponding to each target element in the generated sparse matrix;

[0280] Combine the coordinate information corresponding to each target element in the sparse matrix to determine the position information of the target to be detected.

[0281] In one embodiment, the position determination module 602 is configured to determine the position information of the target to be detected based on the generated sparse matrix according to the following steps:

[0282] Perform at least one convolution process on each target element in the generated sparse matrix based on the trained convolutional neural network to obtain a convolution result;

[0283] Based on the convolution result, determine the position information of the target to be detected.

[0284] Embodiment III

[0285] The present disclosure embodiment also provides an electronic device, as Figure 7 shown, which is a schematic structural diagram of the electronic device provided by the present disclosure embodiment, including: a processor 701, a memory 702, and a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701 (such as Figure 6In the target detection device shown, for the instructions executed by the information acquisition module 601, the position determination module 602, the direction angle determination module 603, and the target detection module 604), when the electronic device is running, the processor 701 communicates with the memory 702 through the bus 703. When the machine-readable instructions are executed by the processor 701, the following processing is performed: obtaining multiple frames of point cloud data scanned by the radar device, and the time information of each frame of point cloud data scanned; based on each frame of point cloud data, determining the position information of the target to be detected; based on the position information of the target to be detected in each frame of point cloud data, determining the scanning direction angle information of the target to be detected scanned by the radar device in each frame of point cloud data; according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information of the target to be detected when scanned by the radar device in each frame of point cloud data, and the time information of each frame of point cloud data scanned, determining the movement information of the target to be detected.

[0286] The embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the target detection method described in the foregoing method embodiments. Among them, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0287] The computer program product of the target detection method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes. The instructions included in the program codes can be used to execute the steps of the target detection method described in the foregoing method embodiments. For details, please refer to the foregoing method embodiments and will not be elaborated here.

[0288] The embodiments of the present disclosure further provide a computer program, which when executed by a processor implements any one of the methods in the foregoing embodiments. The computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0289] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. In several embodiments provided by the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0290] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0291] In addition, in each embodiment of the present disclosure, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0292] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present disclosure. The foregoing storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0293] Finally, it should be noted that the above-described embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting it. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the technical field of the present disclosure can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A target detection method, characterized in that, the method includes: Obtaining multiple frames of point cloud data scanned by a radar device, and the time information of each frame of point cloud data scanned; the time information of each frame of point cloud data scanned includes the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data; Based on each frame of point cloud data, determining the position information of the target to be detected; Based on the position information of the target to be detected in each frame of point cloud data, determining the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data; According to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data, determining the movement information of the target to be detected; The determining the movement information of the target to be detected according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data includes: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to this frame of point cloud data, determining the scanning time information when the target to be detected is scanned in this frame of point cloud data; Based on the position information of the target to be detected in multiple frames of point cloud data, determining the displacement information of the target to be detected; Based on the scanning time information when the target to be detected in the multiple frames of point cloud data is respectively scanned, and the displacement information of the target to be detected, determining the movement speed information of the target to be detected; The for each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to this frame of point cloud data, determining the scanning time information when the target to be detected is scanned in this frame of point cloud data includes: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, and the scanning start angle information in the scanning start and end angle information corresponding to this frame of point cloud data, determining the first angle difference between the direction angle of the target to be detected and the scanning start angle; and, Based on the scanning end angle information in the scanning start and end angle information corresponding to this frame of point cloud data, and the scanning start angle information, determining the second angle difference between the scanning end angle and the scanning start angle; and, Based on the scan termination time information when ending the scan of this frame of point cloud data in the scan start and end time information corresponding to this frame of point cloud data, and the scan start time information when starting to scan this frame of point cloud data in the scan start and end time information corresponding to this frame of point cloud data, determine the time difference between the scan termination time information and the scan start time information; Based on the first angular difference, the second angular difference, the time difference, and the scan start time information, determine the scan time information when the target to be detected in this frame of point cloud data is scanned.

2. The method according to claim 1, wherein, the method further includes: Controlling the intelligent device based on the moving speed information of the target to be detected and the speed information of the intelligent device provided with the radar device.

3. The method according to claim 1, wherein, the method further includes: Predict the movement trajectory of the target to be detected in a future time period based on the movement information and historical movement trajectory information of the target to be detected.

4. The method according to any one of claims 1-3, wherein, the determining the position information of the target to be detected based on each frame of point cloud data includes: Performing rasterization processing on each frame of point cloud data to obtain a raster matrix; the value of each element in the raster matrix is used to represent whether there is a point cloud point at the corresponding raster; Generate a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected; Based on the generated sparse matrix, determine the position information of the target to be detected.

5. The method according to claim 4, wherein, the generating a sparse matrix corresponding to the target to be detected according to the raster matrix and the size information of the target to be detected includes: Performing at least one dilation processing operation or erosion processing operation on the target elements in the raster matrix according to the raster matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected; wherein, the target elements are elements representing that there are point cloud points at the corresponding rasters.

6. The method according to claim 5, wherein, the performing at least one dilation processing operation or erosion processing operation on the target elements in the raster matrix according to the raster matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected includes: Performing at least one shift processing and logical operation processing on the target elements in the raster matrix to obtain a sparse matrix corresponding to the target to be detected, wherein the difference between the coordinate range size of the obtained sparse matrix and the size of the target to be detected is within a preset threshold range.

7. The method according to claim 5, wherein, the performing at least one dilation processing operation on the elements in the raster matrix according to the raster matrix and the size information of the target to be detected to generate a sparse matrix corresponding to the target to be detected includes: Perform a first negation operation on the elements in the grid matrix before the current dilation processing operation to obtain the grid matrix after the first negation operation; Based on a first preset convolution kernel, perform at least one convolution operation on the grid matrix after the first negation operation to obtain a grid matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected; Perform a second negation operation on the elements in the grid matrix with the preset sparsity after at least one convolution operation to obtain the sparse matrix.

8. The method according to claim 7, wherein, the performing a first negation operation on the elements in the grid matrix before the current dilation processing operation to obtain the grid matrix after the first negation operation includes: Based on a second preset convolution kernel, perform a convolution operation on other elements in the grid matrix before the current dilation processing operation except the target element to obtain a first negated element, and based on the second preset convolution kernel, perform a convolution operation on the target element in the grid matrix before the current dilation processing operation to obtain a second negated element; Based on the first negated element and the second negated element, obtain the grid matrix after the first negation operation.

9. The method according to claim 7, wherein, the performing at least one convolution operation on the grid matrix after the first negation operation based on a first preset convolution kernel to obtain a grid matrix with a preset sparsity after at least one convolution operation includes: For the first convolution operation, perform a convolution operation on the grid matrix after the first negation operation and the first preset convolution kernel to obtain the grid matrix after the first convolution operation; Determine whether the sparsity of the grid matrix after the first convolution operation reaches the preset sparsity; If not, loop to perform the step of performing a convolution operation on the grid matrix after the previous convolution operation and the first preset convolution kernel to obtain the grid matrix after the current convolution operation until a grid matrix with a preset sparsity after at least one convolution operation is obtained.

10. The method according to claim 9, wherein, the first preset convolution kernel has a weight matrix and a bias corresponding to the weight matrix; for the first convolution operation, performing a convolution operation on the grid matrix after the first negation operation and the first preset convolution kernel to obtain the grid matrix after the first convolution operation includes: For the first convolution operation, select each grid sub-matrix from the grid matrix after the first negation operation according to the size of the first preset convolution kernel and the preset step length; For each selected grid sub-matrix, perform a multiplication operation on the grid sub-matrix and the weight matrix to obtain a first operation result, and perform an addition operation on the first operation result and the bias to obtain a second operation result; Based on the second operation results corresponding to each grid sub-matrix, determine the grid matrix after the first convolution operation.

11. The method according to claim 4, wherein, Performing at least one erosion processing operation on the elements in the grid matrix according to the grid matrix and the size information of the target to be detected, to generate a sparse matrix corresponding to the target to be detected, including: Performing at least one convolution operation on the grid matrix to be processed based on a third preset convolution kernel, to obtain a grid matrix with a preset sparsity after at least one convolution operation; the preset sparsity is determined by the size information of the target to be detected; Determining the grid matrix with the preset sparsity after the at least one convolution operation as the sparse matrix corresponding to the target to be detected.

12. The method according to claim 4, wherein, Performing rasterization processing on each frame of point cloud data to obtain a grid matrix, including: Performing rasterization processing on each frame of point cloud data to obtain a grid matrix and the corresponding relationship between each element in the grid matrix and the coordinate range information of each point cloud point; The determining the position information of the target to be detected based on the generated sparse matrix includes: Determining the coordinate information corresponding to each target element in the generated sparse matrix based on the corresponding relationship between each element in the grid matrix and the coordinate range information of each point cloud point; Combining the coordinate information corresponding to each of the target elements in the sparse matrix to determine the position information of the target to be detected.

13. The method according to claim 4, wherein, The determining the position information of the target to be detected based on the generated sparse matrix includes: Performing at least one convolution process on each target element in the generated sparse matrix based on a trained convolutional neural network to obtain a convolution result; Determining the position information of the target to be detected based on the convolution result.

14. An object detection device, wherein, The device includes: An information acquisition module, configured to acquire multiple frames of point cloud data scanned by a radar device, and the time information of each frame of point cloud data scanned; the time information of each frame of point cloud data scanned includes the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data; A position determination module, configured to determine the position information of the target to be detected based on each frame of point cloud data; A direction angle determination module, configured to determine the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data based on the position information of the target to be detected in each frame of point cloud data; An object detection module, configured to determine the movement information of the target to be detected according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data; The determining the movement information of the target to be detected according to the position information of the target to be detected in each frame of point cloud data, the scanning direction angle information when the target to be detected is scanned by the radar device in each frame of point cloud data, and the scanning start and end time information and the scanning start and end angle information corresponding to each frame of point cloud data includes: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, as well as the scanning start and end time information and scanning start and end angle information corresponding to this frame of point cloud data, determine the scanning time information when the target to be detected is scanned in this frame of point cloud data; Based on the position information of the target to be detected in multiple frames of point cloud data, determine the displacement information of the target to be detected; Based on the scanning time information when the target to be detected in the multiple frames of point cloud data is respectively scanned, and the displacement information of the target to be detected, determine the moving speed information of the target to be detected; The step of for each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, as well as the scanning start and end time information and scanning start and end angle information corresponding to this frame of point cloud data, determine the scanning time information when the target to be detected is scanned in this frame of point cloud data, includes: For each frame of point cloud data, based on the scanning direction angle information when the target to be detected is scanned in this frame of point cloud data, and the scanning start angle information among the scanning start and end angle information corresponding to this frame of point cloud data, determine the first angle difference between the direction angle of the target to be detected and the scanning start angle; and, Based on the scanning end angle information among the scanning start and end angle information corresponding to this frame of point cloud data, and the scanning start angle information, determine the second angle difference between the scanning end angle and the scanning start angle; and, Based on the scanning end time information when scanning this frame of point cloud data ends in the scanning start and end time information corresponding to this frame of point cloud data, and the scanning start time information when starting to scan this frame of point cloud data in the scanning start and end time information corresponding to this frame of point cloud data, determine the time difference between the scanning end time information and the scanning start time information; Based on the first angle difference, the second angle difference, the time difference, and the scanning start time information, determine the scanning time information when the target to be detected is scanned in this frame of point cloud data.

15. An electronic device, characterized in that, it includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the target detection method according to any one of claims 1 to 13 are executed.

16. A computer-readable storage medium, characterized in that, A computer program is stored on this computer-readable storage medium. When the computer program is run by a processor, the steps of the target detection method according to any one of claims 1 to 13 are executed.

Images