Object detection method and apparatus

Abstract

To provide a method and device for detecting objects.SOLUTION: An object detection method comprises: performing matching between a radar detection result and a video detection result; generating grid cells in a radar coordinate system based on the radar detection result; generating a border frame in an image coordinate system based on the grid cells; and performing object detection based on the border frame.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to the field of object detection technology, and in particular to a method and apparatus for object detection. [Background technology]

[0002] The goal of object detection is to figure out what objects are there and where they are in an image (or frame in a video). Fast and accurate object detection algorithms enable computers to drive cars and enable auxiliary devices to transmit real-time scene information to users and various other applications.

[0003] In recent years, a large amount of related technology has already developed object detection with good performance. However, it still faces challenges in two aspects: the length of processing time and the accuracy of the object's location on the image. Systems such as DPM (Deformable Part Models) use a window sliding method, in which the classifier operates on positions that are uniformly distributed over the entire image. However, in terms of performance, such methods are less efficient and accurate due to the design of the sliding step. State-of-the-art methods such as R-CNN (Region-Convolutional Neural Networks) use a region definition method, which first generates potential boundary boxes in the image, and then runs a classifier on these defined boxes. After classification, post-processing is used to simplify the boundary boxes to remove duplicate detections, and then the boxes are re-evaluated (scored) based on other objects in the scene. However, such complex pipelines run slowly and are difficult to optimize. Summary of the Invention [Problem to be solved by the invention]

[0004] The inventors discovered that the slowness of modern methods such as YOLO (you look only once) can be completely solved by redefining object detection as a regression problem, which uses a single convolutional network to simultaneously predict multiple bounding boxes and class probabilities for these boxes.

[0005] As shown in Figure 1, YOLO first divides the input image into an S×S grid. For each grid cell, it can predict B bounding boxes and the confidence scores of these boxes. Each bounding box consists of five predictors, namely x, y, w, h and confidence. The confidence score includes the confidence of the C class probability of the B bounding box. So, in total, these predictors are encoded as an S×S×(B×5+C) tensor. The network complexity is S 2 Also, considering the processing time, B is also limited because it is determined by S 2 Therefore, one of the problems with YOLO is that its bounding box accuracy is relatively low. However, in reality, the vast majority of grids contain no content, and predicting these empty grid cells is a waste of processing time.

[0006] To address at least one of the above problems or other similar problems, embodiments of the present invention provide an object detection method and apparatus. [Means for solving the problem]

[0007] According to a first aspect of an embodiment of the present invention, there is provided an object detection method, comprising: Matching radar detection results with video detection results; generating grid cells based on the radar detections in a radar coordinate system; generating a bounding box based on the grid cells in an image coordinate system; and Performing object detection based on the bounding box.

[0008] According to a second aspect of an embodiment of the present invention, there is provided an object detection apparatus, comprising: a matching unit for matching radar detection results with video detection results; a first generation unit for generating grid cells based on the radar detection results in a radar coordinate system; a second generating unit for generating a bounding box based on the grid cells in an image coordinate system; and A detection unit performs object detection based on the bounding box.

[0009] According to a third aspect of an embodiment of the present invention, there is provided an image processing device, wherein the image processing device includes a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to realize the above-mentioned object detection method.

[0010] One of the beneficial effects of the embodiment of the present invention is that the present invention can avoid blind search, segmentation and prediction on the complete video frame by indicating the possible location of the bounding box on the video frame based on the radar detection result, which can significantly reduce the processing time of the object detection method (e.g., DPM, R-CNN, etc.) or improve the accuracy of the size and location of the bounding box under the same processing time condition. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates the process of adopting the YOLO method to perform object detection. [Diagram 2] FIG. 2 illustrates a method for detecting objects according to an embodiment of the present invention. [Diagram 3] FIG. 4 is another diagram illustrating an object detection method according to an embodiment of the present invention. [Figure 4] FIG. 1 illustrates an object detection device according to an embodiment of the present invention. [Diagram 5]FIG. 1 is a diagram illustrating an image processing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.

[0013] <Example of the first aspect> An embodiment of the present invention provides an object detection method. Figure 2 is a diagram showing the object detection method in the embodiment of the present invention. As shown in Figure 2, the method includes the following operations (steps):

[0014] Operation 201: Matching radar detection results with video detection results; Operation 202: generating grid cells based on the radar detection results in a radar coordinate system; Operation 203: generating a bounding box based on the grid cells in an image coordinate system; Operation 204: Perform object detection based on the bounding box.

[0015] The present invention avoids blind search, segmentation and prediction on the complete video frame by indicating possible locations of the bounding box on the video frame based on the radar detection results, thereby significantly reducing the processing time of object detection methods (e.g., DPM, R-CNN, etc.) or improving the accuracy of the size and location of the bounding box under the same processing time conditions.

[0016] In an embodiment of the present invention, the radar and the camera simultaneously acquire sensing data asynchronously, the radar captures original data and outputs detection results (referred to as radar detection results), and the camera captures video frames and outputs detection results (referred to as video detection results).

[0017] In an embodiment of the present invention, the radar detection result includes a point cloud, the velocity of each point in the point cloud, and other information. Based on the points in the point cloud, a clustering or tracking result can be obtained using an algorithm, such as Density-Based Spatial Clustering of Applications with Noise (dBscan) or Extended Kalman Filter (EKF). The clustering or tracking result represents the location of the object, which is the coordinate of the object relative to the radar in a (planar) Cartesian coordinate system, represented as (x, y).

[0018] In operation 201, the radar detection result is matched with the video detection result, i.e., the radar frame is matched with the video frame to find the radar frame and the video frame at the same time, or find the radar frame and the video frame whose timestamp difference is smaller than a certain threshold, so that their coordinates (x, y) in the radar coordinate system are calibrated to a position in the video frame, which is represented as (u, v). The radar frame is matched with the video frame to map the point cloud in the radar frame to the video frame, i.e., the position of each point in the radar detection result on the video frame can be determined. Note that the present invention is not limited to a specific matching method, and reference can be made to related art.

[0019] In operation 202, if the point cloud can be clustered into a cluster, i.e., if some points of the point cloud can be clustered into a cluster in the radar frame, the cluster is classified into a cluster having a width (W c i ) and height (H c i), where “i” denotes the i-th cluster, where the points in the cluster lie within a rectangular area, which can be thought of as a grid cell.

[0020] For example, in operation 202, in some embodiments, generating grid cells based on radar detection results includes first performing clustering on the radar detection results to obtain at least one cluster, and then generating grid cells based on coordinates or signal strength of points in the cluster. An example is given below.

[0021] In some embodiments, for each cluster, the coordinates of the center point of the cluster are set as the center of the grid cell, and the difference between the abscissa of the rightmost point in the cluster (i.e., the maximum x-coordinate in the cluster) and the abscissa of the leftmost point in the cluster (i.e., the minimum x-coordinate in the cluster) is set as the width W of the grid cell. c i The difference between the ordinate of the topmost point in the cluster (i.e., the maximum y-coordinate in the cluster) and the ordinate of the bottommost point in the cluster (i.e., the minimum y-coordinate in the cluster) is the height H of the grid cell. c i which gives us the grid cells for each cluster.

[0022] In some embodiments, for each cluster, the coordinates of the point with the maximum signal strength in the cluster are used as the center of the grid cell, and the preset width and height are set as the width W of the grid cell. c i and height H c i Alternatively, the product of the preset width and height and the weight is the width W of the network unit. c i and height H c iThus, a grid cell corresponding to each cluster can be obtained. Here, the preset width and height may be set based on experience, or may be determined based on the distance between the point with the maximum signal strength and the radar, but the present invention is not limited thereto. Alternatively, a fixed width and height may be preset, and a weight of the grid cell may be determined based on other policies or factors (e.g., the distance between the point with the maximum signal strength and the radar), and the product of the weight and the preset fixed width and height may be determined as the width W of the grid cell. c i and height H c i It is also possible to use this method.

[0023] In the above embodiment, different clusters have different grid sizes. The present invention is not limited thereto, and in some embodiments, a unified grid size may be determined for ease of addressing video frames. The unified grid size is determined according to factors such as addressing efficiency, detection accuracy, etc. For different objects, the unified grid size is calculated in different ways.

[0024] For example, to improve the efficiency of object detection, we set a uniform grid size width (W g ) is max(W c i ) and the uniform grid size height (H g ) is max(H c i ) may be equal to

[0025] In addition, for example, in order to obtain more accurate detection results, the width of the grid size (W g ) is min(W c i ) and the uniform grid size height (H g ) is min(H c i ) may be equal to

[0026] Furthermore, the present invention is not limited to a clustering method, and may be the above-mentioned dBscan algorithm or another clustering algorithm, but detailed description thereof will be omitted here.

[0027] In operation 202, if it is difficult to cluster the point cloud into clusters, i.e., if it is not possible to cluster the points in the point cloud in the radar frame, the coordinates of each point in the radar detection result (i.e., each point in the point cloud on the radar frame) are taken as the center of a grid cell, and then the size of the grid cell (i.e., the width W c i and height H c i ) may be confirmed.

[0028] For example, the size of a grid cell having the coordinates of each point as the center of the grid cell is determined based on the distance between each point and the radar. In some embodiments, the size of a grid cell corresponding to a point close to the radar may be relatively large, and the size of a grid cell corresponding to a point far from the radar may be relatively small. N The size may be determined empirically, but the invention is not limited in this regard.

[0029] Also, for example, based on the signal strength of each point, the size of the grid cell with the coordinate of the point as the center of the grid cell is determined. Since the strength of the signal is affected by multiple factors, generally speaking, a strong signal means that the dielectric constant of the medium to be measured is large and the reflection area is large. Therefore, in some embodiments, the size of the corresponding grid cell may be relatively large for a point with a high signal strength, and conversely, the size of the corresponding grid cell may be relatively small for a point with a low signal strength. The specific size may be determined based on experience, and the present invention is not limited thereto.

[0030] In the embodiment of the present invention, the distance between each point in the point cloud of the radar frame and the radar may be calculated from the radar detection result, but the detailed description thereof will be omitted here. In addition, in the embodiment of the present invention, the calculation method, detection method, or determination method of the signal strength of each point is not limited, and may be obtained by any conventional method, but the detailed description thereof will be omitted here.

[0031] In addition, similar to the method when the point cloud can be clustered into clusters as described above, even if it is not possible to cluster the point cloud, a single unified grid size may be determined, and since the method is similar to the method when the point cloud can be clustered into clusters as described above, a detailed description thereof will be omitted here.

[0032] In an embodiment of the present invention, grid cells are obtained based on radar detections (points in a point cloud) and areas that do not contain points are ignored, so that the calibration positions (grid cells) from the radar frames indicate possible locations of a bounding box on the video frame, which can be predicted based on the grid cells.

[0033] In operation 203, in some embodiments, for each grid cell, at least one frame covering the grid cell can be generated as a bounding frame corresponding to the grid cell by sliding left / right or up / down along the u-axis and v-axis of a center point of the grid cell on the video frame.

[0034] In the present invention, the unit of sliding is not limited, and for example, a bounding frame covering the above-mentioned grid cell may be obtained by using the size (width and height) of the grid cell as a unit and sliding left and right or up and down along the u-axis and v-axis of the center point of the grid cell, or a bounding frame covering the above-mentioned grid cell may be obtained by using a preset unit size as a unit and sliding left and right or up and down along the u-axis and v-axis of the center point of the grid cell.

[0035] In some embodiments, the location of the bounding box may be represented by the coordinates of the upper left corner of the bounding box, and the size of the bounding box may be represented by the width and height of the bounding box, although the invention is not limited thereto and other quantities may be used to represent the location and size of the bounding box, depending on the means by which the bounding box is obtained or other factors.

[0036] In some embodiments, the reliability of the bounding box may be determined, for example, based on the number of points in the grid cell, in which case the reliability of the bounding box is determined based on the number of points in the grid cell, and in some embodiments, the more points in the grid cell generating the bounding box, the more likely an object exists and the more reliable the bounding box corresponding to the grid cell is. Also, for example, the reliability of the bounding box may be determined based on the signal strength of the points in the grid cell, and in some embodiments, the stronger the signal strength of the point with the highest signal strength in the grid cell generating the bounding box, the more likely an object exists and the more reliable the bounding box corresponding to the grid cell is.

[0037] In an embodiment of the present invention, the velocity of a point in the point cloud on the radar frame can further provide a priori information for classifying an object in a video frame, that is, the object detection method of the present invention can further determine a possible classification of an object (referred to as object type) based on the velocity of a point in the radar detection result, so that object detection can be performed based on the bounding box obtained in the above-mentioned operation 203 and the possible classification corresponding to each point in the radar detection result. For example, if the velocity of a point is close to zero, it means that the point represents a static object in an intelligent road intersection scene, such as a lattice fence, a fire hydrant, a road sign, etc., and if not, it means that the point represents a moving object, such as a car, a motorcycle, a bicycle, or a pedestrian. The priori information can improve the efficiency of object detection.

[0038] In operation 204, based on the above-mentioned bounding box, or based on the above-mentioned bounding box and the above-mentioned a priori information, an algorithm known to those skilled in the art can be adopted to obtain an object detection result, which is not limited to the algorithm used in the present invention and may be any artificial intelligence (AI) processing method, such as the YOLO method.

[0039] According to the method of the embodiment of the present invention, by indicating possible positions of the bounding box on the video frame based on the radar detection result, blind search, segmentation and prediction on the complete video frame can be avoided, thereby significantly reducing the processing time of the object detection method (e.g., DPM, R-CNN, etc.) or improving the accuracy of the size and position of the bounding box under the same processing time conditions.

[0040] Although the method of the embodiment of the present invention has been described above based on Fig. 2, the present invention is not limited thereto, and when specifically implemented, the execution order between each operation may be appropriately adjusted, for example, the execution order of operation 201 and operation 202 may be exchanged, or some operations may be added or removed. That is, a person skilled in the art may appropriately modify the above content based on the description of Fig. 2 above.

[0041] 3 is another diagram of an object detection method according to an embodiment of the present invention, showing the whole process from obtaining radar original data and camera video frames to outputting object detection results. As shown in FIG. 3, the method includes the following operations (steps):

[0042] Operation 301: Obtaining radar original data by a radar; Operation 302: Perform signal processing on the radar original data to obtain the object's position relative to the radar and the object's velocity; Operation 303: generating grid cells based on the point cloud on the radar frame; Operation 304: acquiring a video frame by a camera; Operation 305: performing calibration on the radar frame and the video frame and mapping the above-mentioned grid cells onto the video frame; Operation 306: performing time synchronization on the radar frames and the video frames; Operation 307: indicating possible locations of a bounding box based on grid cells; Operation 308: Constraining classification typology using velocities of points in the point cloud on the radar frame; Operation 309: Perform AI processing to obtain an object detection result.

[0043] Although the method of the embodiment of the present invention has been described above with reference to FIG. 3, the present invention is not limited thereto, and when specifically implemented, the execution order between each operation may be appropriately adjusted, for example, operation 301 and operation 304 may be performed simultaneously and asynchronously, or the execution order of operation 303 and operations 305-306 may be exchanged, for example, operations 305 and 306 may be performed first, and then operation 303 may be performed. In addition, some operations may be increased or decreased, for example, operation 308 may be omitted. That is, a person skilled in the art may appropriately modify the above content based on the description of FIG. 3 above.

[0044] Also, in the example of Fig. 3, operation 303 corresponds to operation 202 in Fig. 2, and operation 307 corresponds to operation 203 in Fig. 2. Since operations 202 and 203 have been described in detail in the previous embodiment, detailed description thereof will be omitted here.

[0045] Although only the steps or processes according to the present invention have been described above, the present invention is not limited thereto. The method may further include other steps or processes, and the specific contents of these steps or processes can be referred to in the prior art.

[0046] As described above, the method according to the embodiment of the present invention can significantly reduce the processing time of the object detection method, or improve the accuracy of the size and position of the bounding box under the same processing time condition.

[0047] <Example of the second aspect> The embodiment of the present invention provides an object detection device, the principle of which the device solves the problem is similar to that of the embodiment of the first aspect, so that the specific implementation of the device can be referred to the embodiment of the first aspect, and the detailed description thereof will be omitted here.

[0048] 4 is a diagram illustrating an object detection apparatus in an embodiment of the present invention, as shown in Fig. 4, the object detection apparatus 400 in the embodiment of the present invention includes a matching unit 401, a first generation unit 402, a second generation unit 403 and a detection unit 404. The matching unit 401 performs matching between the radar detection result and the video detection result; the first generation unit 402 generates grid cells in a radar coordinate system according to the radar detection result; the second generation unit 403 generates a bounding box in an image coordinate system according to the grid cells; and the detection unit 404 is used to perform object detection based on the bounding box.

[0049] In some embodiments, as shown in FIG. 4, the object detection apparatus 400 in the embodiments of the present invention further includes a determination unit 405, which determines an object type based on the velocity of the points in the radar detection result, and the detection unit 404 performs object detection based on the above-mentioned bounding box and the object type of each point in the radar detection result.

[0050] In some embodiments, the first generation unit 402 performs clustering on the radar detection results to obtain at least one cluster; for each cluster, a grid cell for each cluster is obtained by taking the coordinate of the center point of the cluster as the center of the grid cell, taking the difference between the abscissa of the rightmost point and the abscissa of the leftmost point in the cluster as the width of the grid cell, and taking the difference between the ordinate of the topmost point and the ordinate of the bottommost point in the cluster as the height of the grid cell.

[0051] In some embodiments, the first generation unit 402 performs clustering on the radar detection results to obtain at least one cluster; for each cluster, the coordinates of the point with the maximum signal strength in the cluster are taken as the center of the grid cell, and the preset width and height are taken as the width and height of the grid cell, or the multiplication of the preset width and height with the weight is taken as the width and height of the network unit, thereby obtaining a grid cell for each cluster.

[0052] In some embodiments, the first generation unit 402 may determine the size of a grid cell based on the coordinates of each point in the radar detection result as the center of the grid cell, and based on the distance between the point and the radar, determine whether the grid cell is large or small with the coordinates of the point as the center of the grid cell.

[0053] In some embodiments, the first generation unit 402 may determine the coordinates of each point in the radar detection result as the center of a grid cell, and determine the size of the grid cell with the coordinates of the point as the center based on the signal strength of the point.

[0054] In some embodiments, for each grid cell, the second generation unit 403 can generate at least one frame that covers the grid cell as a corresponding bounding frame for the grid cell by sliding left / right or up / down along the u-axis and v-axis of a center point of the grid cell.

[0055] In some embodiments, the second generation unit 403 may further determine a reliability of the bounding box based on the number of points in the grid cell, or may determine a reliability of the bounding box based on the signal strength of the points in the grid cell.

[0056] In some embodiments, the second generation unit 403 generates the bounding box by sliding left and right or up and down along the u and v axes of the center point of the grid cell in units of the width and height of the grid cell, or the second generation unit 403 can generate the bounding box by sliding left and right or up and down along the u and v axes of the center point of the grid cell in units of a preset unit size.

[0057] Although only the components or modules according to the present invention have been described above, the present invention is not limited thereto. The object detection device 400 may further include other components or modules, and the specific contents of these components or modules can be found in the related art.

[0058] As can be seen from the above embodiments, the object detection apparatus in the embodiments of the present invention can significantly reduce the processing time of the object detection method, or improve the accuracy of the size and position of the bounding box under the same processing time conditions.

[0059] <Example of the third aspect> An embodiment of the present invention provides an image processing device, which may be, for example, a computer, a server, a workstation, a notebook computer, a smartphone, etc., but the embodiment of the present invention is not limited thereto.

[0060] 5 is a diagram showing an image processing device in an embodiment of the present invention, and as shown in Fig. 5, the image processing device 500 in the embodiment of the present invention includes at least one interface (not shown in Fig. 5), a processor (e.g., a central processing unit (CPU)) 501, and a memory 502, and the memory 502 is connected to the processor 501. The memory 502 can store various data, and can also store a program 503 for object detection, and execute the program 503 under the control of the processor 501, and can store various preset values ​​and predetermined conditions, etc.

[0061] In one embodiment, the function of the object detection device 400 described in the embodiment of the second aspect may be integrated into a processor 501 to realize the object detection method described in the embodiment of the first aspect. For example, the processor 501 may be configured as follows: Matching radar detection results with video detection results; generating grid cells based on the radar detections in a radar coordinate system; generating a bounding box based on the grid cells in an image coordinate system; and Object detection is performed based on the bounding box.

[0062] In another embodiment, the object detection device 400 described in the embodiment of the second aspect may be configured separately from the processor 501, for example, the object detection device 400 may be configured as a chip connected to the processor 501, and the functions of the object detection device 400 may be realized under the control of the processor 501.

[0063] It should be noted that the image processing device 500 may further include a display 505 and an I / O device 504, or may not necessarily include all the components shown in Fig. 5, for example, it may further include a camera head (not shown) for obtaining input image frames. Also, the image processing device 500 may further include components not shown in Fig. 5, for which reference can be made to related art.

[0064] In an embodiment of the present invention, the processor 501 may be referred to as a controller or an operation controller, and may include a microprocessor or other processing and / or logic device. The processor 501 can receive inputs and control the operation of the components of the image processing device 500.

[0065] In an embodiment of the present invention, the storage unit 502 may be one or more of a buffer, a flash memory, a HDD, a removable medium, a volatile storage unit, a non-volatile storage unit, or other suitable devices, and may store various information and programs for processing. The processor 501 may execute the programs stored in the storage unit 502 to store and process information. The functions of the other components are similar to those of the conventional components, and therefore detailed descriptions thereof are omitted here. Each component of the image processing device 500 may be realized by dedicated hardware, firmware, software, or a combination thereof, all of which are within the scope of the present invention.

[0066] By performing object detection using an image processing device according to an embodiment of the present invention, the processing time of the object detection method can be significantly reduced, or the accuracy of the size and position of the bounding box can be improved under the same processing time conditions.

[0067] An embodiment of the present invention further provides a computer readable program, which, when executed in an image processing device, causes the image processing device to perform a method according to the first aspect of the embodiment.

[0068] An embodiment of the present invention further provides a storage medium having stored thereon a computer readable program, wherein the computer readable program causes an image processing device to perform the method according to the first aspect of the embodiment.

[0069] The apparatus or method described with reference to the embodiments of the present invention may be realized by hardware, software modules executed by a processor, or a combination of both. For example, one or more functions in the functional block diagram and / or one or more combinations of functions in the functional block diagram may correspond to each software module in a computer program or each hardware module. Also, each of these software modules may correspond to each step shown in the diagrams illustrating the method. These hardware modules may be realized by solidifying these software modules using, for example, an FPGA (field-programmable gate array).

[0070] In addition, the apparatus, method, etc. according to the embodiments of the present invention may be realized by software, hardware, or a combination of hardware and software. The present invention also relates to such a computer readable program, i.e., the program, when executed by a logic component, can cause the logic component to realize the above-mentioned apparatus or component, or cause the logic component to realize the above-mentioned method or steps thereof. Furthermore, the present invention also relates to a storage medium, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc., storing the above-mentioned program.

[0071] Furthermore, the following supplementary notes are disclosed regarding the above-mentioned embodiments.

[0072] (Appendix 1) 1. A method for object detection, comprising: Matching radar detection results with video detection results; generating grid cells based on the radar detections in a radar coordinate system; generating a bounding box based on the grid cells in an image coordinate system; and performing object detection based on the bounding box.

[0073] (Appendix 2) 2. The method according to claim 1, comprising: The method further includes determining an object type based on the velocity of points in the radar detection. The method, wherein performing object detection based on the bounding box includes performing object detection based on the bounding box and an object type of each point in the radar detection result.

[0074] (Appendix 3) 2. The method according to claim 1, comprising: generating grid cells based on the radar detections in a radar coordinate system, Performing clustering on the radar detection results to obtain at least one cluster; and determining, for each cluster, a grid cell corresponding to each cluster by defining the coordinates of the center point of the cluster as the center of the grid cell, defining the difference between the abscissa of the rightmost point and the ordinate of the leftmost point in the cluster as the width of the grid cell, and defining the difference between the ordinate of the topmost point and the ordinate of the bottommost point in the cluster as the height of the grid cell.

[0075] (Appendix 4) 2. The method according to claim 1, comprising: generating grid cells based on the radar detections in a radar coordinate system, Performing clustering on the radar detection results to obtain at least one cluster; and The method includes obtaining a grid cell corresponding to each cluster by, for each cluster, taking the coordinates of the point with the maximum signal strength in the cluster as the center of the grid cell, and taking the preset width and height as the width and height of the grid cell, or taking the product of the preset width and height and a weight as the width and height of the network unit.

[0076] (Appendix 5) 2. The method according to claim 1, comprising: generating grid cells based on the radar detections in a radar coordinate system, The method includes determining whether a grid cell having the coordinates of each point in the radar detection result as the center of the grid cell is large or small based on a distance between the point and the radar.

[0077] (Appendix 6) 2. The method according to claim 1, comprising: generating grid cells based on the radar detections in a radar coordinate system, The method includes determining whether a grid cell having the coordinates of each point in the radar detection result as the center of the grid cell is large or small based on the signal strength of the point.

[0078] (Appendix 7) 2. The method according to claim 1, comprising: Generating a bounding box based on the grid cells in an image coordinate system includes: generating, for each grid cell, at least one frame that covers the grid cell as a bounding frame corresponding to the grid cell by sliding left / right or up / down along a u-axis and a v-axis of a center point of the grid cell.

[0079] (Appendix 8) 8. The method according to claim 7, Generating a bounding box based on the grid cells in an image coordinate system further includes: determining a confidence level of the bounding box based on a number of points in the grid cell; or determining a confidence level of the bounding box based on signal strength of points in the grid cells.

[0080] (Appendix 9) 8. The method according to claim 7, Sliding left and right or up and down along the u and v axes of the center point of the grid cell sliding left and right or up and down along the u and v axes of the center point of the grid cell in units of the width and height of the grid cell; or sliding left / right or up / down along u and v axes of a center point of said grid cell in units of a preset unit size.

[0081] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and any modification to the present invention falls within the technical scope of the present invention as long as it does not depart from the spirit of the present invention.

Claims

1. 1. An object detection device, comprising: a matching unit for matching the radar detection result with the video detection result; a first generation unit for generating grid cells based on the radar detections in a radar coordinate system, clustering the radar detection results to obtain at least one cluster; When the at least one cluster is obtained, For each cluster, obtaining a grid cell for the cluster by using the coordinates of a center point of the cluster as the center of the grid cell, or obtaining a grid cell for the cluster by using the coordinates of a point in the cluster where the signal strength is maximum as the center of the grid cell; if the radar detections cannot be clustered into at least one cluster, generating grid cells centered on the coordinates of each point in the radar detection results; First generation unit; a second generating unit for generating a bounding box based on the grid cells in an image coordinate system; and a detection unit for performing object detection based on the bounding box.

2. The object detection device according to claim 1 , A determination unit is further included for determining an object type based on a speed of a point in the radar detection result; The detection unit performs object detection based on the bounding box and the object type of each point in the radar detection result.

3. The object detection device according to claim 1 , The first generation unit obtains a grid cell corresponding to each cluster by setting the coordinates of the center point of the cluster as the center of the grid cell, setting the difference between the abscissa of the rightmost point and the abscissa of the leftmost point in the cluster as the width of the grid cell, and setting the difference between the ordinate of the topmost point and the ordinate of the bottommost point in the cluster as the height of the grid cell.

4. The object detection device according to claim 1 , The first generation unit obtains a grid cell corresponding to each cluster by setting, for each cluster, the coordinates of the point in the cluster where the signal strength is maximum as the center of the grid cell, and setting the preset width and height as the width and height of the grid cell, or setting the product of the preset width and height and the weight as the width and height of the network unit, an object detection device.

5. An object detection device as claimed in claim 1, The object detection device, when the radar detection results cannot be clustered into at least one cluster, the first generation unit determines the size of a grid cell centered on the coordinates of each point in the radar detection results based on the distance between each point and the radar, and generates grid cells centered on the coordinates of each point in the radar detection results by making the size of the grid cell relatively large corresponding to points close to the radar and making the size of the grid cell relatively small corresponding to points far from the radar.

6. An object detection device as claimed in claim 1, The object detection device, when the radar detection results cannot be clustered into at least one cluster, the first generation unit determines the size of a grid cell with the coordinates of each point as the center of the grid cell based on the signal strength of each point, and generates grid cells centered on the coordinates of each point in the radar detection results by making the size of the grid cell corresponding to points with high signal strength relatively large and the size of the grid cell corresponding to points with weak signal strength relatively small.

7. The object detection device according to claim 1 , The first generation unit defines the coordinates of each point in the radar detection result as the center of a grid cell, and determines the size of a grid cell with the coordinates of the point as the center based on the distance between the point and the radar.

8. The object detection device according to claim 1 , The first generation unit determines the size of a grid cell having the coordinates of each point in the radar detection result as the center of the grid cell based on the signal strength of the point.

9. The object detection device according to claim 1 , An object detection device, wherein for each grid cell, the second generation unit generates at least one frame covering the grid cell as a bounding frame corresponding to the grid cell by sliding left and right or up and down along the u-axis and v-axis of the center point of the grid cell.

10. The object detection device according to claim 9, The second generation unit further determines a reliability of the bounding box based on a number of points in the grid cell, or determines a reliability of the bounding box based on a signal strength of the points in the grid cell.

11. The object detection device according to claim 9, The second generation unit slides left and right or up and down along the u and v axes of the center point of the grid cell using the width and height of the grid cell as units, or the second generation unit slides left and right or up and down along the u and v axes of the center point of the grid cell using a preset unit size as units.

12. An image processing device including a processor and a memory, The storage device stores a computer program, The processor executes the computer program to Matching radar detection results with video detection results; generating grid cells based on the radar detections in a radar coordinate system; clustering the radar detection results to obtain at least one cluster; When the at least one cluster is obtained, For each cluster, obtaining a grid cell for the cluster by using the coordinates of a center point of the cluster as the center of the grid cell, or obtaining a grid cell for the cluster by using the coordinates of a point in the cluster where the signal strength is maximum as the center of the grid cell; if the radar detections cannot be clustered into at least one cluster, generating grid cells centered on the coordinates of each point in the radar detection results; thing; generating a bounding box based on the grid cells in an image coordinate system; detecting objects based on said bounding boxes; 2. An image processing device configured to:

Images