Obstacle speed detection method and device

Abstract

The present application provides an obstacle speed detection method and apparatus. The obstacle speed detection method includes: acquiring radar data collected by a radar and video data collected by a camera; performing matching processing on the radar data and the video data; based on the weight features of the radar data and the weight features of the video data, fusing the radar data and the video data that have undergone matching processing to determine obstacle orientation information; and determining the obstacle speed based on a target time threshold and the obstacle orientation information corresponding to the target time threshold. The obstacle speed detection method and apparatus provided by the present application match the radar data and the video data, fuse them based on the weight features of both, determine the obstacle orientation information according to the fusion result, and thus obtain the obstacle speed within the target time threshold, making the confidence level more objective and capable of improving the accuracy of obstacle speed detection.

Description

Technical Field

[0001] This application relates to the field of driverless technology, and particularly to a method and device for detecting the speed of an obstacle. Background Art

[0002] With the development of vehicle technology, vehicles are becoming more and more inclined to be information-based and intelligent. Taking driverless technology as an example, it is necessary to detect obstacles that appear in front of the vehicle, obtain the speed of the obstacles, so as to control the vehicle or remind the driver for assisted driving.

[0003] Currently, using a single sensor to detect the speed of dangerous targets in the driving area in front of the vehicle often faces problems of high false detection rate and high missed detection rate. The amount of information obtained by using a single sensor to implement the method of assisted driving is small, the recognition result is inaccurate, and it cannot adapt to complex highway environments. Summary of the Invention

[0004] This application provides a method and device for detecting the speed of an obstacle, so as to make the confidence level of obstacle speed detection more objective and improve the accuracy of obstacle speed detection.

[0005] This application provides a method for detecting the speed of an obstacle. The method for detecting the speed of an obstacle includes: obtaining radar data collected by a radar and video data collected by a camera; performing matching processing on the radar data and the video data; based on the weight characteristics of the radar data and the weight characteristics of the video data, fusing the radar data and the video data that have undergone matching processing to determine the obstacle azimuth information; based on a target time threshold and the obstacle azimuth information corresponding to the target time threshold, determining the obstacle speed.

[0006] According to a method for detecting the speed of an obstacle provided by this application, the weight characteristic of the radar data is obtained based on a first detection speed obtained by the radar detecting an obstacle sample and a speed label corresponding to the obstacle sample; the weight characteristic of the video data is obtained based on a second detection speed obtained by the camera detecting the obstacle sample and a speed label corresponding to the obstacle sample.

[0007] According to a method for detecting the speed of an obstacle provided by this application, the weight characteristic of the radar data is a first covariance matrix corresponding to the first detection speed and the speed label; the weight characteristic of the video data is a second covariance matrix corresponding to the second detection speed and the speed label.

[0008] A method for detecting the speed of an obstacle provided by the present application, which determines the speed of the obstacle based on the azimuth information of the obstacle and a preset time threshold, includes: obtaining the displacement characteristics of the obstacle from the azimuth information of the obstacle based on the preset time threshold; and obtaining the speed of the obstacle based on the displacement characteristics of the obstacle and the preset time threshold.

[0009] A method for detecting the speed of an obstacle provided by the present application, which performs matching processing on the radar data and the video data, includes: performing spatial calibration processing on the radar data and the video data; and performing time synchronization processing on the radar data and the video data.

[0010] A method for detecting the speed of an obstacle provided by the present application, which performs spatial calibration processing on the radar data and the video data, includes: setting the coordinate systems of the radar data and the video data to coincide.

[0011] A method for detecting the speed of an obstacle provided by the present application, which performs time synchronization processing on the radar data and the video data, includes: aligning each time point of the radar data with each time point of the video data.

[0012] The present application also provides an obstacle speed detection device, which includes: an acquisition module for acquiring radar data collected by a radar and video data collected by a camera; a matching module for performing matching processing on the radar data and the video data; a fusion module for fusing the radar data and the video data that have undergone matching processing based on the weight characteristics of the radar data and the weight characteristics of the video data to determine the azimuth information of the obstacle; and a determination module for determining the speed of the obstacle based on a target time threshold and the azimuth information of the obstacle corresponding to the target time threshold.

[0013] According to an obstacle speed detection device provided by the present application, the determination module includes: a first determination sub-module for obtaining the displacement characteristics of the obstacle from the azimuth information of the obstacle based on a preset time threshold; and a second determination sub-module for obtaining the speed of the obstacle based on the displacement characteristics of the obstacle and the preset time threshold.

[0014] According to an obstacle speed detection device provided by the present application, the matching module includes: a first matching sub-module for performing spatial calibration processing on the radar data and the video data; and a second matching sub-module for performing time synchronization processing on the radar data and the video data.

[0015] The present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned obstacle speed detection methods are implemented.

[0016] The present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned obstacle speed detection methods are implemented.

[0017] The obstacle speed detection method provided by the present application matches radar data and video data, fuses them based on their weight characteristics, determines the obstacle azimuth information according to the fusion result, and thus obtains the speed of the obstacle within the target time threshold, making the confidence level of the obstacle speed detection more objective and improving the accuracy of the obstacle speed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for the implementation examples or the description of the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is one of the flow diagrams of the obstacle speed detection method provided by the present application;

[0020] Figure 2 is another flow diagram of the obstacle speed detection method provided by the present application;

[0021] Figure 3 is yet another flow diagram of the obstacle speed detection method provided by the present application;

[0022] Figure 4 is the structural diagram of the obstacle speed detection device provided by the present application;

[0023] Figure 5 is the structural diagram of the matching module of the obstacle speed detection device provided by the present application;

[0024] Figure 6 is the structural diagram of the determination module of the obstacle speed detection device provided by the present application;

[0025] Figure 7 is the structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the objectives, technical solutions and advantages of this application clearer, the following will describe the technical solutions in this application clearly and completely in conjunction with the accompanying drawings in this application. Apparently, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without making creative efforts shall fall within the scope of protection of this application.

[0027] The following will describe Figures 1 - 5 the obstacle speed detection method and device of this application.

[0028] It should be noted that this obstacle speed detection method uses a radar and a camera in combination to detect obstacles in front of the vehicle. The radar and the camera can be installed at the front of the vehicle, for example, they can be installed at the position of the vehicle bumper.

[0029] This obstacle speed detection method can be applicable to various types of vehicles. For example, the vehicle can be a car, a truck, a logistics vehicle, a garbage truck, a road sweeper, or other vehicles with a walking function. The obstacles in front of the vehicle can be moving objects such as motor vehicles, pedestrians, bicycles, electric vehicles or animals, and can also be traffic facilities or municipal public facilities such as guardrails, signal lamp posts or roadside trees.

[0030] As Figure 1 shown, this application provides an obstacle speed detection method, and this obstacle speed detection method includes the following steps 110-step 140.

[0031] Among them, step 110: Obtain the radar data collected by the radar and the video data collected by the camera.

[0032] The radar can be: lidar, millimeter wave radar or ultrasonic radar, etc.

[0033] Here, taking the millimeter wave radar as an example, the millimeter wave radar is a radar that detects in the millimeter wave band (millimeter wave). The working frequency band is generally 24 GHz to 300 GHz, and the wavelength is 1 to 10 mm, which is between microwave and centimeter wave. It accurately detects the direction and distance of the target by emitting electromagnetic waves to the obstacle and receiving the echo, and it has all-weather and all-time as well as accurate speed measurement and ranging. It has some advantages of microwave radar and optoelectronic radar. Compared with ultrasonic radar, the millimeter wave radar has the characteristics of small volume, light weight and high spatial resolution. Compared with optical sensors such as infrared, laser and camera, the millimeter wave radar has strong ability to penetrate fog, smoke and dust and has the characteristics of all-weather and all-time. In addition, the anti-interference ability of the millimeter wave radar is also better than other vehicle-mounted sensors. The working frequencies of the millimeter wave radar applied to vehicles can be 24 GHz and 77 GHz.

[0034] At the same time, lidar is also a very important sensor in the field of autonomous driving. Lidar uses lasers to detect targets. By scanning at 600 or 1200 revolutions per minute, it can obtain very detailed real-time three-dimensional point cloud data, including the three-dimensional coordinates, distance, azimuth angle, intensity of the reflected laser, laser coding, time, etc. of the target. Commonly used ones include single-line, 4-line, 16-line, 32-line, 64-line, and 128-line bundles. It is a high-precision sensor with good stability and high robustness. However, the cost of lidar is relatively high. In addition, lasers are greatly affected by the atmosphere and meteorology. Atmospheric attenuation and bad weather reduce the operating range, and atmospheric turbulence reduces the measurement accuracy of lidar. It is difficult to search for and capture targets when the laser beam is narrow. Generally, other devices are used to implement large-airspace and fast coarse target capture first, and then the lidar is used to perform precise tracking and measurement on the target.

[0035] The cameras can be: monocular cameras, binocular stereo vision cameras, panoramic vision cameras, and infrared cameras. Here, the monocular camera is taken as an example. The monocular camera can capture videos in front of the vehicle.

[0036] Monocular cameras are mainly used for the detection and recognition of feature symbols, such as lane line detection, traffic sign recognition, traffic light recognition, pedestrian and vehicle detection, etc. Although the reliability of visual detection is not very high at present, visual computing based on machine learning will definitely be an essential part when autonomous driving becomes popular.

[0037] Both the radar and the camera are oriented towards the front area of the vehicle. When installing the radar and the camera, the field of view of the radar and the field of view of the camera can be set to coincide, that is, to ensure that their acquisition ranges are roughly the same.

[0038] The radar can collect radar data containing obstacles, and the camera can collect video data containing obstacles. The formats of the radar data and the video data are continuous files in the order of time points.

[0039] In practical applications, the radar emits electromagnetic waves externally, receives the radar signals reflected by obstacles, and processes the radar signals to obtain radar data in the form of traces that change with time. This radar data contains the distance, azimuth, and pitch values of the obstacles. The radar signals can be preprocessed. For example, the measurement sets obtained from multiple scans can be associated to obtain the tracks of the obstacles, and the errors of the radar data can be corrected through filtering algorithms and data processing.

[0040] The camera can capture multiple frames of images of the obstacles. As time goes by, the multiple frames of images form video data. Video data refers to a continuous sequence of images. In essence, it is composed of a group of continuous images. For the images themselves, there is no structural information except for the order in which they appear.

[0041] Step 120: Perform matching processing on the radar data and the video data.

[0042] It can be understood that the formats of the radar data and the video data are different. Here, the formats of the radar data and the video data are adjusted. For example, the size of each frame of the radar data and the video data can be scaled so that the radar data and the video data can be correspondingly matched, and the radar data and the video data intercepted at a certain moment can be used to describe the same obstacle.

[0043] Step 130: Based on the weight features of the radar data and the weight features of the video data, fuse the radar data and the video data that have undergone matching processing to determine the obstacle orientation information.

[0044] It can be understood that the weight features of the radar data and the weight features of the video data have been determined during the vehicle debugging. The weight features of the radar data are related to the physical characteristics of the radar itself and can represent the recognition accuracy of the radar. The weight features of the video data are related to the physical characteristics of the camera and can represent the recognition accuracy of the camera.

[0045] Here, the radar data and the video data that have undergone matching processing are put together, and weights are assigned to the radar data and the video data according to their weight features to determine their respective proportions. For example, the radar data accounts for 60% and the video data accounts for 40%. At this time, when fusing the radar data and the video data, the corresponding traces of the radar data and the corresponding pixel points of the video data can be searched and matched and then fused and output. In other words, the radar data is composed of multiple traces, and the video data is composed of multiple pixel points. Here, the multiple traces of the radar data and the multiple pixel points of the video data are correspondingly matched according to the time sequence and the spatial sequence, so as to fuse the radar data and the video data to obtain the fused data. For example, the feature points in the radar data can be weighted at a ratio of 60%, and the feature points in the video data can be weighted at a ratio of 40%. Then, the feature points of the radar data after weighting processing and the feature points of the radar data after weighting processing are superimposed, that is, the respective feature points in the radar data and the respective feature points in the video data are superimposed according to the weight features to obtain the fused data, such as 60%x + 40%y = z, where x represents the feature points of the radar data, y represents the feature points of the video data, and z represents the feature points of the fused data.

[0046] The obstacle orientation information is included in the fused data. The obstacle orientation information may include the position of the obstacle and the moving direction of the obstacle.

[0047] Step 140: Determine the obstacle speed based on the target time threshold and the obstacle azimuth information corresponding to the target time threshold.

[0048] It can be understood that the obstacle azimuth information has the obstacle azimuth information at different time points. The obstacle azimuth information may include the position of the obstacle and the moving direction of the obstacle. Here, a preset target time threshold is given. The target time threshold is a time period, and the obstacle azimuth information within this time period is found in the obstacle azimuth information.

[0049] The change value of the obstacle azimuth information within the target time threshold can be divided by the target time threshold to obtain the obstacle speed.

[0050] Preset the target time threshold, for example, it can be 0.01s. The obstacle speed can be determined according to the change of the obstacle azimuth information within this 0.01s. The obstacle speed includes the moving direction of the obstacle and the absolute value of the moving speed.

[0051] For example, when the vehicle starts, it is regarded as the 0 moment. At the 0 moment, the azimuth information of the obstacle includes: the position is (0, 0), and the direction is: due north. At the 0.01s moment, the azimuth information of the obstacle includes: the position is (0, 0.2), and the direction is due north. Then the obstacle speed can be (0.2 - 0) / 0.01 = 20m / s.

[0052] After the speed of the obstacle is recognized, the host of the vehicle can make corresponding judgments based on the current vehicle speed and the obstacle speed, and then control the direction of the vehicle, or can give a voice prompt to the driver or display it on the in-vehicle display screen to remind the driver to make adjustments in time to avoid safety accidents.

[0053] It should be noted that the millimeter-wave radar mainly obtains the distance, speed and angle of the target object by sending electromagnetic waves to the target object and receiving the echo. The vision solution is slightly more complex. The monocular camera needs to first perform target recognition, and then estimate the distance of the target according to the pixel size of the target in the image. The camera solution has a low cost, can identify different objects, and has advantages in aspects such as the measurement accuracy of the object height and width, the recognition accuracy of lane lines and pedestrian recognition, and is an indispensable sensor for realizing functions such as lane departure warning and traffic sign recognition. However, the operating distance and ranging accuracy are not as good as those of the millimeter-wave radar, and it is easily affected by factors such as light and weather. The millimeter-wave radar is less affected by light and weather factors and has high ranging accuracy, but it is difficult to identify elements such as lane lines and traffic signs. In addition, the millimeter-wave radar can achieve higher-precision target speed detection through the principle of Doppler shift.

[0054] It is worth mentioning that the radar device itself has a confidence level, which is obtained based on the physical characteristics of the radar device. The camera device also has a confidence level, which is related to parameters such as the type of obstacle, the frame rate of the camera, and the acquisition frequency of the camera. The confidence level of the fusion scheme for obtaining obstacle data by fusing radar data and video data is as follows: the confidence levels of the camera and the radar are summed. If the sum is greater than or equal to 1, the confidence level of the fusion scheme is 1. If the sum is less than 1, the confidence level of the fusion scheme is equal to the sum. Through experimental verification, it is found that the confidence level of this fusion scheme for obtaining obstacle data by fusing radar data and video data is more objective, and the obstacle azimuth information obtained after fusing radar data and video data can better reflect the true movement of the obstacle, can eliminate the measurement errors of the radar and the camera respectively, and is more accurate in detecting the speed of the obstacle.

[0055] For the scheme of measuring the obstacle speed by the radar alone, the first obstacle azimuth feature is directly extracted from the radar data, and the first obstacle speed is calculated based on the first obstacle azimuth feature. For the scheme of measuring the obstacle speed by the camera alone, the second obstacle azimuth feature is directly extracted from the video data, and the second obstacle speed is calculated based on the second obstacle azimuth feature. If the first obstacle speed and the second obstacle speed are simply averaged directly, some information in the first obstacle azimuth feature and the second obstacle azimuth feature will be lost, resulting in the underutilization of the radar data and the video data. The fusion scheme of this application directly fuses the radar data and the video data. The obstacle azimuth information obtained after fusion completely retains the radar data and the video data, can fully utilize the important information in the radar data and the video data, and can better reflect the true movement of the obstacle, making the detection of the obstacle speed more accurate.

[0056] The obstacle speed detection method provided by this application matches the radar data and the video data, fuses them based on their weight features, determines the obstacle azimuth information according to the fusion result, and thus obtains the obstacle speed within the target time threshold, making the confidence level of the obstacle speed detection more objective and improving the accuracy of the obstacle speed detection.

[0057] As Figure 2 shown, in some embodiments, step 120 above: performing matching processing on the radar data and the video data includes the following steps 121 - 122.

[0058] Among them, step 121: performing spatial calibration processing on the radar data and the video data.

[0059] It can be understood that the spatial calibration process refers to adjusting the spatial states of radar data and video data to a unified reference system, and the coordinate systems of radar data and video data can be set to coincide.

[0060] It can be understood that the coordinate conversion relationships can be established among the precise radar coordinate system, the three-dimensional world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system. Setting the coordinate systems of radar data and video data to coincide means converting the measurement values in different coordinate systems to the same coordinate system. Since the video data collected by the camera is mainly visual, the measurement points in the radar coordinate system can be converted to the corresponding pixel coordinate system of the camera through coordinate transformation. According to the above conversion relationships, the conversion relationship between the radar coordinate system and the camera pixel coordinate system can be obtained, and thus the coordinate systems of radar data and video data can be set to coincide.

[0061] Step 122: Perform time synchronization processing on the radar data and video data.

[0062] It can be understood that the time synchronization process refers to aligning the time sequence representations of radar data and video data, that is, aligning each time point of radar data with each time point of video data.

[0063] It can be understood that the radar data and video data can achieve time synchronization by synchronously collecting data through the camera and the radar in time. For example, the sampling period of the millimeter-wave radar is 50 ms, that is, the sampling frame rate is 20 frames per second, while the sampling frame rate of the camera is 25 frames per second. To ensure the reliability of the data, based on the sampling rate of the camera, every time the camera captures a frame of image, the data cached in the previous frame of the millimeter-wave radar is selected, that is, the time alignment of a frame of radar data and video data is completed together, thus ensuring the time synchronization of radar data and video data.

[0064] In some embodiments, the weight feature of the radar data is obtained based on the first detection speed detected by the radar for the obstacle sample and the speed label corresponding to the obstacle sample; the weight feature of the video data is obtained based on the second detection speed detected by the camera for the obstacle sample and the speed label corresponding to the obstacle sample.

[0065] It can be understood that the target sample and the speed label corresponding to the target sample can be obtained; based on the detection of the target sample by the radar, the first detection speed is obtained, and based on the detection of the target sample by the camera, the second detection speed is obtained; based on the first detection speed and the speed label, the weight feature of the radar data is obtained, and based on the second detection speed and the speed label, the weight feature of the video data is obtained.

[0066] In other words, during vehicle debugging, the radar and camera are tested to simulate multiple target samples, each target sample having a corresponding speed label, and the speed label can be the true speed value of the target sample.

[0067] The radar is used to detect the target sample to obtain a first detection speed, and the first detection speed is compared with the speed label to obtain the weight feature of the radar. The weight feature of the radar characterizes the recognition accuracy of the radar. At the same time, the camera is used to detect the target sample to obtain a second detection speed, and the second detection speed is compared with the speed label to obtain the weight feature of the camera. The weight feature of the camera characterizes the recognition accuracy of the camera.

[0068] In some embodiments, the weight feature of the radar data is the first covariance matrix corresponding to the first detection speed and the speed label; the weight feature of the video data is the second covariance matrix corresponding to the second detection speed and the speed label.

[0069] It can be understood that covariance is used in probability theory and statistics to measure the overall error between two variables. Variance is a special case of covariance, that is, when the two variables are the same.

[0070] Covariance represents the overall error between two variables, which is different from variance that only represents the error of one variable. If the change trends of two variables are the same, that is, if one of them is greater than its own expected value and the other is also greater than its own expected value, then the covariance between the two variables is positive. If the change trends of two variables are opposite, that is, if one of them is greater than its own expected value and the other is less than its own expected value, then the covariance between the two variables is negative.

[0071] The covariance matrix can be simply understood as follows. Let the column vector random variables X and Y be respectively m and n scalar elements. The covariance between these two variables is defined as an m×n matrix, where X contains variables X1, X2,......, Xm, and Y contains variables Y1, Y2,......, Yn. Assume that the expected value of X1 is μ1 and the expected value of Y2 is μ2. Then the element (1,2) in the covariance matrix is the covariance between X1 and Y2. The covariance matrix can clearly show the correlation between two variables.

[0072] As Figure 3 shown, in some embodiments, step 140: determining the obstacle speed based on the obstacle azimuth information and a preset time threshold includes the following steps 141-step 142.

[0073] Step 141: Obtain the obstacle displacement feature from the obstacle azimuth information based on the preset time threshold.

[0074] It is understandable that a preset time threshold, that is, a preselected time period, is first determined, and data corresponding to the time period is intercepted from the obstacle orientation information, and the obstacle displacement feature is selected from this part of the data. The obstacle displacement feature characterizes the displacement of the obstacle within the preset time threshold, and the obstacle displacement feature may include the position change of the obstacle and the moving direction of the obstacle.

[0075] Step 142: Based on the obstacle displacement feature and the preset time threshold, obtain the speed of the obstacle.

[0076] It is understandable that the displacement of the obstacle characterized by the obstacle displacement feature within the preset time threshold can be divided by the preset time threshold to obtain the speed of the obstacle.

[0077] The obstacle speed detection device provided by the present application will be described below. The obstacle speed detection device described below can be correspondingly referred to the obstacle speed detection method described above.

[0078] As Figure 4 shown, an embodiment of the present application further provides an obstacle speed detection device, and the obstacle speed detection device includes: an acquisition module 410, a matching module 420, a fusion module 430, and a determination module 440.

[0079] Among them, the acquisition module 410 is used to acquire radar data collected by a radar and video data collected by a camera.

[0080] The matching module 420 is used to perform matching processing on the radar data and the video data.

[0081] The fusion module 430 is used to fuse the radar data and the video data that have undergone matching processing based on the weight features of the radar data and the weight features of the video data to determine the obstacle orientation information.

[0082] The determination module 440 is used to determine the speed of the obstacle based on the target time threshold and the obstacle orientation information corresponding to the target time threshold.

[0083] In some embodiments, the weight feature of the radar data is obtained based on the first detection speed obtained by the radar detecting an obstacle sample and the speed label corresponding to the obstacle sample; the weight feature of the video data is obtained based on the second detection speed obtained by the camera detecting the obstacle sample and the speed label corresponding to the obstacle sample.

[0084] In some embodiments, the weight feature of the radar data is the first covariance matrix corresponding to the first detection speed and the speed label; the weight feature of the video data is the second covariance matrix corresponding to the second detection speed and the speed label.

[0085] AsFigure 5 As shown, in some embodiments, the matching module 420 includes: a first matching sub-module 421 and a second matching sub-module 422.

[0086] The first matching sub-module 421 is used to perform spatial calibration processing on radar data and video data.

[0087] The second matching sub-module 422 is used to perform time synchronization processing on radar data and video data.

[0088] In some embodiments, the first matching sub-module 421 is further used to set the coordinate systems of the radar data and the video data to coincide.

[0089] In some embodiments, the second matching sub-module 422 is further used to align each time point of the radar data and each time point of the video data.

[0090] As Figure 6 As shown, in some embodiments, the determination module 440 includes: a first determination sub-module 441 and a second determination sub-module 442.

[0091] The first determination sub-module 441 is used to obtain the obstacle displacement feature from the obstacle azimuth information based on a preset time threshold;

[0092] The second determination sub-module 442 is used to obtain the obstacle speed based on the obstacle displacement feature and the preset time threshold.

[0093] The obstacle speed detection device provided by the embodiments of the present application is used to execute the above-mentioned obstacle speed detection method. Its specific implementation manner is consistent with the implementation manner recorded in the method embodiment, and the same beneficial effects can be achieved, which will not be elaborated here.

[0094] Figure 7 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 7As shown in the figure, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logical instructions in the memory 730 to execute the obstacle speed detection method, and the obstacle speed detection method includes: obtaining radar data collected by a radar and video data collected by a camera; performing matching processing on the radar data and the video data; based on the weight characteristics of the radar data and the weight characteristics of the video data, fusing the radar data and the video data that have undergone matching processing to determine the obstacle azimuth information; based on the target time threshold and the obstacle azimuth information corresponding to the target time threshold, determining the obstacle speed.

[0095] The processor 710 may also execute to execute the obstacle speed detection method. The determining the obstacle speed based on the obstacle azimuth information and a preset time threshold includes: obtaining the obstacle displacement characteristics from the obstacle azimuth information based on the preset time threshold; obtaining the obstacle speed based on the obstacle displacement characteristics and the preset time threshold.

[0096] The processor 710 may also execute to execute the obstacle speed detection method. The performing matching processing on the radar data and the video data includes: performing spatial calibration processing on the radar data and the video data; performing time synchronization processing on the radar data and the video data.

[0097] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0098] The processor 710 in the electronic device provided by the embodiment of the present application can call the logic instructions in the memory 730 to implement the above-mentioned obstacle speed detection method. The specific implementation manner is consistent with the method implementation manner and can achieve the same beneficial effects, which will not be elaborated here.

[0099] On the other hand, the present application also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the obstacle speed detection method provided by the above-mentioned various methods. The obstacle speed detection method includes: acquiring radar data collected by a radar and video data collected by a camera; performing matching processing on the radar data and the video data; based on the weight characteristics of the radar data and the weight characteristics of the video data, fusing the radar data and the video data that have undergone matching processing to determine the obstacle azimuth information; and determining the obstacle speed based on the target time threshold and the obstacle azimuth information corresponding to the target time threshold.

[0100] Meanwhile, the present application also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the obstacle speed detection method provided by the above-mentioned various methods. Determining the obstacle speed based on the obstacle azimuth information and a preset time threshold includes: acquiring the obstacle displacement characteristics from the obstacle azimuth information based on the preset time threshold; and obtaining the obstacle speed based on the obstacle displacement characteristics and the preset time threshold.

[0101] Meanwhile, the present application also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the obstacle speed detection method provided by the above-mentioned various methods. Performing matching processing on the radar data and the video data includes: performing spatial calibration processing on the radar data and the video data; and performing time synchronization processing on the radar data and the video data.

[0102] When the computer program product provided by the embodiment of the present application is executed, it implements the above-mentioned obstacle speed detection method. The specific implementation manner is consistent with the method implementation manner and can achieve the same beneficial effects, which will not be elaborated here.

[0103] In another aspect, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the obstacle speed detection method provided above. The obstacle speed detection method includes: obtaining radar data collected by a radar and video data collected by a camera; performing matching processing on the radar data and the video data; based on the weight characteristics of the radar data and the weight characteristics of the video data, fusing the radar data and the video data that have undergone matching processing to determine the obstacle orientation information; and determining the obstacle speed based on the target time threshold and the obstacle orientation information corresponding to the target time threshold.

[0104] Meanwhile, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the obstacle speed detection method provided above. Determining the obstacle speed based on the obstacle orientation information and a preset time threshold includes: obtaining the obstacle displacement characteristics from the obstacle orientation information based on the preset time threshold; and obtaining the obstacle speed based on the obstacle displacement characteristics and the preset time threshold.

[0105] Meanwhile, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the obstacle speed detection method provided above. The performing matching processing on the radar data and the video data includes: performing spatial calibration processing on the radar data and the video data; and performing time synchronization processing on the radar data and the video data.

[0106] When the computer program stored on the non-transitory computer-readable storage medium provided by the embodiment of the present application is executed, it implements the above obstacle speed detection method. Its specific implementation manner is consistent with the method implementation manner and can achieve the same beneficial effects, which will not be elaborated here.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting the speed of an obstacle, characterized in that, it includes: Obtaining radar data collected by a radar and video data collected by a camera; Performing matching processing on the radar data and the video data; Based on the weight characteristics of the radar data and the weight characteristics of the video data, fusing the radar data and the video data that have undergone matching processing to determine the obstacle orientation information; the fusion includes weighting the feature points in the radar data according to the radar data proportion, weighting the feature points in the video data according to the video data proportion, and superimposing the feature points of the radar data after weighting processing and the feature points of the video data after weighting processing; The radar data proportion and the video data proportion are determined by weighting the radar data and the video data according to the weight characteristics of the radar data and the weight characteristics of the video data; Based on the target time threshold and the obstacle orientation information corresponding to the target time threshold, determining the obstacle speed; The weight characteristic of the radar data is obtained based on the first detection speed obtained by the radar detecting the obstacle sample and the speed label corresponding to the obstacle sample; The weight characteristic of the video data is obtained based on the second detection speed obtained by the camera detecting the obstacle sample and the speed label corresponding to the obstacle sample; The speed label is the true speed value of the obstacle sample; Wherein, the weight characteristic of the radar data is the first covariance matrix corresponding to the first detection speed and the speed label; the weight characteristic of the video data is the second covariance matrix corresponding to the second detection speed and the speed label.

2. The method for detecting the speed of an obstacle according to claim 1, characterized in that, The determining the obstacle speed based on the obstacle orientation information and a preset time threshold includes: Based on the preset time threshold, obtaining the obstacle displacement characteristic from the obstacle orientation information; Based on the obstacle displacement characteristic and the preset time threshold, obtaining the obstacle speed.

3. The method for detecting the speed of an obstacle according to any one of claims 1-2, characterized in that, The performing matching processing on the radar data and the video data includes: Performing spatial calibration processing on the radar data and the video data; Performing time synchronization processing on the radar data and the video data.

4. The method for detecting the speed of an obstacle according to claim 3, characterized in that, The performing spatial calibration processing on the radar data and the video data includes: Setting the coordinate system of the radar data and the coordinate system of the video data to coincide.

5. The method for detecting the speed of an obstacle according to claim 3, characterized in that, The performing time synchronization processing on the radar data and the video data includes: Aligning each time point of the radar data and each time point of the video data.

6. An obstacle speed detection device, characterized in that, it includes: An acquisition module for acquiring radar data collected by a radar and video data collected by a camera; A matching module for performing matching processing on the radar data and the video data; A fusion module for fusing the radar data and the video data after matching processing based on the weight features of the radar data and the weight features of the video data to determine obstacle azimuth information; the fusion includes weighting the feature points in the radar data according to the radar data ratio, weighting the feature points in the video data according to the video data ratio, and superimposing the feature points of the weighted radar data and the feature points of the weighted video data; The radar data ratio and the video data ratio are determined by weighting the radar data and the video data according to the weight features of the radar data and the weight features of the video data; A determination module for determining the obstacle speed based on a target time threshold and the obstacle azimuth information corresponding to the target time threshold; The weight feature of the radar data is obtained based on the first detection speed obtained by the radar detecting the obstacle sample and the speed label corresponding to the obstacle sample; The weight feature of the video data is obtained based on the second detection speed obtained by the camera detecting the obstacle sample and the speed label corresponding to the obstacle sample; The speed label is the true speed value of the obstacle sample; The weight feature of the radar data is the first covariance matrix corresponding to the first detection speed and the speed label; the weight feature of the video data is the second covariance matrix corresponding to the second detection speed and the speed label.

7. The obstacle speed detection device according to claim 6, wherein, the determination module includes: A first determination sub-module for obtaining the obstacle displacement feature from the obstacle azimuth information based on a preset time threshold; A second determination sub-module for obtaining the obstacle speed based on the obstacle displacement feature and the preset time threshold.

8. The obstacle speed detection device according to claim 6, wherein, the matching module includes: A first matching sub-module for performing spatial calibration processing on the radar data and the video data; A second matching sub-module for performing time synchronization processing on the radar data and the video data.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the program, the steps of the obstacle speed detection method according to any one of claims 1 to 5 are implemented.

10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the obstacle speed detection method according to any one of claims 1 to 5 are implemented.

Images