Sensor detection methods, devices and vehicles

Abstract

A sensor detection method, apparatus, and vehicle are disclosed. The method includes: acquiring at least one image frame captured by a first camera device within a first time period, and a point cloud data set captured by a first radar within the same time period; determining a first slope of the lane line of a first road on which the vehicle is located in a vehicle coordinate system based on the at least one image frame, wherein the vehicle includes the first camera device and the first radar; determining a second slope of the curb of the first road in the vehicle coordinate system based on the point cloud data set; and determining a first angle by which the yaw angle of the first radar deviates based on the first and second slopes. This technical solution can be applied to intelligent vehicles such as electric vehicles and new energy vehicles, and can quickly and accurately detect whether the yaw angle of the first radar in the vehicle has deviated from its calibration value. When the first angle is greater than or equal to a preset threshold, the user can be prompted to perform maintenance and / or the use of the vehicle's intelligent driving functions can be restricted, which helps to improve vehicle safety.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more specifically, to a sensor detection method, apparatus, and vehicle. Background Technology

[0002] As vehicles become increasingly intelligent, the requirements for the accuracy and reliability of sensor perception results are also rising. In the field of intelligent driving, radar sensors (such as lidar) are key sensors for real-time perception of surrounding elements (such as roads, lanes, traffic lights, signs, etc.), and high-precision calibration of these sensors is a prerequisite for achieving their high-precision perception functions.

[0003] However, in actual vehicle use, if external forces such as collisions cause a significant deviation between the actual yaw angle of the radar sensor and the calibrated yaw angle at the vehicle end, the radar sensor's perception results will be inaccurate. In the process of controlling vehicle movement based on the radar sensor's perception results, inaccurate perception results may lead to unexpected vehicle steering or false triggering of automatic emergency braking (AEB), posing safety risks to the vehicle.

[0004] Therefore, a sensor detection scheme that can effectively detect whether the yaw angle of a radar sensor is accurate is urgently needed. Summary of the Invention

[0005] This application provides a sensor detection method, apparatus, and vehicle that can quickly and accurately detect whether the yaw angle of the radar in the vehicle has deviated from the calibration value, which helps to improve vehicle safety.

[0006] In one aspect, a sensor detection method is provided, which can be performed by a vehicle, for example, by the vehicle's computing platform, or by a chip or circuitry used in the vehicle.

[0007] The method includes: acquiring at least one frame of image captured by a first camera device within a first time period, and a point cloud data set captured by a first radar within the first time period, wherein the vehicle includes the first camera device and the first radar; determining a first slope of the lane line of the first road where the vehicle is located in a first coordinate system based on the at least one frame of image, wherein the first coordinate system is a vehicle coordinate system or a world coordinate system; determining a second slope of the curb of the first road in the first coordinate system based on the point cloud data set; and determining a first angle offset by the yaw angle of the first radar based on the first slope and the second slope.

[0008] In some implementations, the first radar is a lidar; or, the first radar is a millimeter-wave radar, such as a four-dimensional (4D) millimeter-wave radar; or, the first radar can also be other radars with altitude measurement capabilities.

[0009] In the above technical solution, the yaw angle of the first radar is verified by using images collected by the camera device. This method has low computational resource consumption, good scene versatility, and is not limited by the initial value. As a result, the coverage of the detectable yaw angle of the first radar is larger and the timeliness is faster.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, at least one frame of image includes an Mth frame of image, the point cloud data set includes an Nth frame of point cloud data, and the time difference between the start time of acquiring the Mth frame of image and the start time of acquiring the Nth frame of point cloud data is less than or equal to a first threshold; determining the first slope of the lane line of the first road where the vehicle is located in the first coordinate system based on at least one frame of image includes: determining the first slope based on the Mth frame of image; determining the second slope of the curb of the first road in the first coordinate system based on the point cloud data set includes: determining the second slope based on the Nth frame of point cloud data.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, determining the second slope based on the Nth frame point cloud data includes: determining a first set of data based on the Nth frame point cloud data, the first set of data indicating the position of the point cloud data corresponding to the road edge in the Nth frame point cloud data in a first coordinate system; clustering and fitting the first set of data to obtain multiple road edge data segments; fitting the slope of each road edge data segment in the multiple road edge data segments to obtain multiple slope values, each slope value being a corresponding slope in the multiple road edge data segments; and determining the second slope based on the multiple slope values.

[0012] Since road edges may not be straight in practice, segmenting, clustering, and fitting the data corresponding to the road edge (or curb) to obtain multiple slope values ​​can more accurately reflect the slope of the road edge at different locations. Furthermore, determining the slope of the road edge based on multiple slope values ​​helps improve the reliability of the determined road edge slope, thereby improving the accuracy of the determined first radar yaw angle offset.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining the first angle offset by the yaw angle of the first radar based on the first slope and the second slope includes: when the standard deviation corresponding to multiple slope values ​​is less than or equal to a preset threshold, determining the first angle based on the first slope and the second slope.

[0014] In the above technical solution, when multiple slope values ​​are not significantly different, it can be determined that the curb of the first road is straight or approximately straight. In this case, the accuracy of the yaw angle of the first radar determined based on the curb slope and lane line slope is more reliable.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, at least one frame of image and point cloud data set is collected during the vehicle's travel in the first road. Determining the first angle offset by the yaw angle of the first radar based on the first slope and the second slope includes: when the angle between the lane line and the vehicle's travel direction indicated by the first slope is less than or equal to a first angle threshold, determining the first angle based on the first slope and the second slope.

[0016] In the above technical solution, when the angle between the lane line and the vehicle's driving direction is less than or equal to a certain threshold, the vehicle can be considered to be driving in a direction parallel to the lane. In this case, the computational complexity required to determine the curb slope is lower, and the reliability of the yaw angle offset angle of the first radar determined based on the curb slope and the lane line slope is higher.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: when the first angle is greater than or equal to the second included angle threshold, the control prompting device prompts the first information, the first information being used to indicate that the pose of the first radar is abnormal and / or to inspect the first radar.

[0018] In the above technical solution, when the yaw angle of the first radar deviates too much, a first message is displayed, which helps to prompt the user to inspect the first radar and eliminate the abnormality as soon as possible.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: restricting the activation of intelligent driving functions when the first angle is greater than or equal to a second included angle threshold.

[0020] If the yaw angle of the first radar deviates excessively, activating the intelligent driving function may lead to unexpected steering of the vehicle or cause the AEB (Autonomous Emergency Braking) to be mistakenly triggered, posing a safety risk. Therefore, in the above technical solution, limiting the activation of the intelligent driving function helps to improve vehicle safety.

[0021] Secondly, a sensor detection device is provided, comprising an acquisition unit and a processing unit, wherein the acquisition unit is configured to: acquire at least one frame of image captured by a first camera device within a first time period, and a point cloud data set captured by a first radar within the first time period, wherein the vehicle includes the first camera device and the first radar; the processing unit is configured to: determine, based on the at least one frame of image, a first slope of the lane line of the first road on which the vehicle is located in a first coordinate system, the first coordinate system including a vehicle coordinate system or a world coordinate system; determine, based on the point cloud data set, a second slope of the curb of the first road in the first coordinate system; and determine, based on the first slope and the second slope, a first angle by which the yaw angle of the first radar is offset.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, at least one frame of image includes the Mth frame of image, the point cloud data group includes the Nth frame of point cloud data, and the time difference between the start time of acquiring the Mth frame of image and the start time of acquiring the Nth frame of point cloud data is less than or equal to a first threshold; the processing unit is used to: determine a first slope based on the Mth frame of image; and determine a second slope based on the Nth frame of point cloud data.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is used to: determine a first set of data based on the Nth frame point cloud data, the first set of data indicating the position of the point cloud data corresponding to the road edge in the Nth frame point cloud data in a first coordinate system; cluster and fit the first set of data to obtain multiple road edge data segments; perform slope fitting on each road edge data segment in the multiple road edge data segments to obtain multiple slope values, each slope value being a corresponding slope in the multiple road edge data segments; and determine a second slope based on the multiple slope values.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is used to: determine the first angle based on the first slope and the second slope when the standard deviation corresponding to multiple slope values ​​is less than or equal to a preset threshold.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, at least one frame of image and point cloud data group are collected during the vehicle's travel in the first road, and the processing unit is used to: determine the first angle based on the first slope and the second slope when the angle between the first slope indicating the lane line and the vehicle's travel direction is less than or equal to the first angle threshold.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: when the first angle is greater than or equal to the second included angle threshold, control the prompting device to prompt the first information, the first information being used to indicate that the pose of the first radar is abnormal and / or to inspect the first radar.

[0027] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is also used to: restrict the activation of intelligent driving functions when the first angle is greater than or equal to the second included angle threshold.

[0028] Thirdly, a sensor detection device is provided, the device comprising: a processor for executing a computer program stored in the memory, such that the device performs the method in any possible implementation of the first aspect described above.

[0029] In conjunction with the third aspect, in some implementations of the third aspect, the device also includes a memory.

[0030] Fourthly, a computer program product is provided, comprising: computer program code, which, when executed on a computer or processor, causes the computer or processor to perform the method in any possible implementation of the first aspect.

[0031] It should be noted that the above computer program code can be stored in whole or in part on a storage medium, which can be packaged together with the processor or packaged separately from the processor.

[0032] Fifthly, a computer-readable storage medium is provided, the computer-readable medium storing instructions that, when executed by a processor, cause the processor to implement the method in any possible implementation of the first aspect.

[0033] In a sixth aspect, a chip is provided that includes circuitry for performing the method in any of the possible implementations of the first aspect described above.

[0034] In a seventh aspect, a vehicle is provided that includes means as in any possible implementation of the second to third aspects, or the vehicle includes computer-readable storage as in any possible implementation of the fifth aspect, or the vehicle includes a chip as in any possible implementation of the sixth aspect, or the vehicle is loaded with computer program code as in any possible implementation of the fourth aspect.

[0035] In conjunction with the seventh aspect, in some implementations of the seventh aspect, the vehicle is a vehicle in a broad sense, such as a means of transportation (e.g., commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (e.g., forklifts, trailers, tractors, etc.), engineering vehicles (e.g., excavators, bulldozers, cranes, etc.), agricultural equipment (e.g., lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. In practical implementation, the vehicle can also be a road vehicle, a water vehicle, an air vehicle, industrial equipment, agricultural equipment, or other intelligent driving equipment such as entertainment equipment.

[0036] For the beneficial effects not described in detail in aspects two through seven, please refer to the description in aspect one, which will not be repeated here. Attached Figure Description

[0037] Figure 1 This is a functional schematic block diagram of the vehicle provided in the embodiments of this application;

[0038] Figure 2 This is a schematic block diagram of the sensor detection system architecture provided in the embodiments of this application;

[0039] Figure 3 This is a schematic flowchart of the sensor detection method provided in the embodiments of this application;

[0040] Figure 4 This is a schematic diagram illustrating the extraction of lane line features from images captured by a camera device, provided in an embodiment of this application.

[0041] Figure 5 This is a schematic diagram showing the position of the lane lines in the vehicle coordinate system according to an embodiment of this application;

[0042] Figure 6 This is another schematic flowchart of the sensor detection method provided in the embodiments of this application;

[0043] Figure 7 This is a schematic diagram of the data selection of the required grid and the detection results provided in the embodiments of this application;

[0044] Figure 8 This is a schematic diagram illustrating an application scenario of the sensor detection method provided in the embodiments of this application;

[0045] Figure 9 This is a schematic flowchart of a sensor detection method provided in an embodiment of this application;

[0046] Figure 10 This is a schematic block diagram of the sensor detection device provided in the embodiments of this application;

[0047] Figure 11 This is another schematic block diagram of the sensor detection device provided in the embodiments of this application. Detailed Implementation

[0048] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0049] Figure 1 This is a functional block diagram of a vehicle provided in an embodiment of this application. For example... Figure 1As shown, the vehicle 100 may include a perception system 120 and a computing platform 150. The perception system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global navigation satellite system (GNSS), such as the global positioning system (GPS), BeiDou system, etc. Alternatively, the perception system 120 may also include one or more of the following: an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0050] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the corresponding functions. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to implement the corresponding functions.

[0051] The computing platform 150 can control the operation of the intelligent driving system, which may include an advanced driving assistance system (ADAS) and an autonomous driving system (ADS). The intelligent driving system utilizes various sensors on the vehicle (including but not limited to: LiDAR, millimeter-wave radar, cameras, ultrasonic sensors, GPS, and inertial measurement units) to acquire information from the vehicle's surroundings, and analyzes and processes this information to achieve functions such as obstacle perception, target recognition, vehicle localization, path planning, and driver monitoring / alerts, thereby improving the safety, automation, and comfort of driving.

[0052] At different levels of autonomous driving (or intelligent driving levels, ranging from L0 to L5, totaling six levels), intelligent driving systems can achieve different levels of automated driving assistance based on artificial intelligence algorithms and information acquired by multiple sensors. These levels of autonomous driving are based on the classification standards of the Society of Automotive Engineers (SAE). Specifically, L0 is no automation; L1 is driver assistance; L2 is partial automation; L3 is conditional automation; L4 is high automation; and L5 is full automation. At levels L1 to L3, the task of monitoring road conditions and reacting is jointly completed by the driver and the system, requiring the driver to take over dynamic driving tasks. Levels L4 and L5 allow the driver to completely transform into a passenger. Currently, the functions that intelligent driving systems can achieve mainly include, but are not limited to: adaptive cruise control, automatic emergency braking, automatic parking, blind spot monitoring, forward cross-traffic alert / braking, rear cross-traffic alert / braking, forward collision warning, lane departure warning, lane keeping assist, rear collision warning, traffic sign recognition, traffic jam assist, and highway assist. It should be understood that the above-mentioned functions can have specific modes at different levels of autonomous driving (L0-L5). The higher the level of autonomous driving, the more intelligent the corresponding mode.

[0053] In this application, when the image acquired by the camera device in the perception system 120 includes lane line information, the computing platform 150 can determine the slope of the lane line in the vehicle coordinate system based on the image. Furthermore, the computing platform 150 can extract curb features from the point cloud data acquired by the lidar in the perception system 120 and determine the curb slope based on these features. Further, the computing platform 150 can determine whether the lidar's yaw angle matches the calibration value based on the lane line slope and the curb slope.

[0054] Figure 2A schematic block diagram of a sensor detection system architecture provided in an embodiment of this application is shown. The system includes a sensing module 210, a detection module 220, and a control module 230. In some implementations, the system also includes a prompting module 240. Specifically:

[0055] The sensing module 210 may include Figure 1 The perception system 120 shown includes one or more camera devices and one or more lidar sensors for collecting environmental information about the area where the vehicle is located, such as parking line information and road edge information. The perception module 210 can send the collected images and point cloud data to the detection module 220.

[0056] The detection module 220 may include Figure 1 One or more processors in the computing platform 150 shown are used to detect whether the yaw angle of the LiDAR has shifted. The detection module 220 includes a lane line detection module 221, a curb feature extraction module 222, and a result verification and statistics module 223. The lane line detection module 221 is used to determine the slope of the lane lines relative to the vehicle coordinate system based on images acquired by the camera device; the curb feature extraction module 222 is used to determine the slope of the road edge relative to the vehicle coordinate system based on point cloud data acquired by the LiDAR or 4D millimeter-wave radar; the result verification and statistics module 223 is used to determine the difference between the actual yaw angle and the calibrated yaw angle of the LiDAR or 4D millimeter-wave radar based on the statistical values ​​of multiple frames of detection results.

[0057] Control module 230 may include Figure 1 One or more processors in the computing platform 150 shown are used to restrict the use of intelligent driving functions when the difference between the actual yaw angle and the calibrated yaw angle of the lidar is greater than or equal to a difference threshold.

[0058] The prompting module 240 may include one or more of the following: an in-vehicle display device, an in-vehicle sound device (such as a speaker, audio system, etc.), and an in-vehicle lighting device (such as ambient lighting associated with sensors). It is used to prompt that the LiDAR or 4D millimeter-wave radar is malfunctioning when the difference between the actual yaw angle and the calibrated yaw angle is greater than or equal to a difference threshold, allowing the user to promptly inspect the LiDAR or 4D millimeter-wave radar. For example, the in-vehicle display device may include an in-vehicle display screen and a projection display screen. The in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as a digital instrument cluster display screen, a central control screen, a display screen in front of the front passenger (also known as the front-seat passenger), a display screen in front of the left rear passenger, and a display screen in front of the right rear passenger; even the vehicle window can be used as a display screen. The projection display screen may include a head-up display, also known as a head-up display system. It is mainly used to display driving information such as speed and navigation on a display device in front of the driver (e.g., the windshield). To reduce driver eye movement time and avoid pupil changes caused by eye movement, HUDs improve driving safety and comfort. Examples of HUDs include combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems.

[0059] It should be understood that the above module is only an example, and in actual applications, it may be added or removed as needed. For example, Figure 2 In the system architecture shown, the prompting module 240 and the detection module 220 are combined into one module.

[0060] The above combination Figure 2 The system involved in the embodiments of this application has been introduced. The following details the system based on... Figure 2 The sensor detection method performed by the system shown.

[0061] Figure 3 This paper illustrates an exemplary flowchart of a sensor detection method provided in an embodiment of this application. The method 300 can be... Figure 2 The lane line detection module 221 shown executes the method 300, which includes:

[0062] S301, acquire the image captured by the camera device, and extract the pixel points corresponding to multiple lane lines based on the image.

[0063] For example, the camera device can be a camera device installed outside the vehicle cabin, such as any one of a front-view camera device, a rear-view camera device, or a surround-view camera device used to acquire images of the front or rear of the vehicle. The front-view camera device can be installed at the windshield; the rear-view camera device can be installed at the trunk; the surround-view camera device includes four cameras installed around the vehicle, and the images acquired by the four cameras are stitched together to obtain a panoramic image of the vehicle's surroundings.

[0064] For example, while the vehicle is in motion, the camera device captures a video stream, and the aforementioned image may include a frame from the video stream.

[0065] S302 determines the position and slope of each lane line in the vehicle coordinate system based on the pixels corresponding to the lane lines and the external parameters of the camera device.

[0066] For example, a frame of image captured by the vehicle's forward-facing camera can be as follows: Figure 4 As shown in (a), the image includes pixels of lane lines 401, 402, and 403, as well as pixels of curbs 404 and 405. Furthermore, the image also includes pixels of other vehicles. Further, the lane-related pixels are extracted using an image recognition algorithm, for example... Figure 4 In (b), lines A to E are considered as lane lines 401 to 403, and the positions of the pixels corresponding to curbs 404 and 405 in the image.

[0067] Furthermore, based on the extrinsic and intrinsic parameters of the camera device, the position of each pixel in the image within the vehicle coordinate system can be determined. It is understood that the intrinsic parameters are the camera device's own parameters, such as focal length. The extrinsic parameters are parameters related to the camera device's mounting position, such as pitch, roll, and yaw angles.

[0068] For example, the intrinsic parameter matrix K of the camera device can be as shown in the following formula (1):

[0069]

[0070] Among them, f x It can be expressed in pixels to describe the length of the focal length along the x-axis in the image coordinate system, f y It can represent the length of the focal length along the y-axis in the image coordinate system using pixels, p x and p y It can be used to represent the position of a pixel in the image coordinate system, with the unit being pixel.

[0071] Based on the intrinsic parameter matrix of the camera device, the coordinates in the pixel coordinate system can be converted to the coordinates in the camera device coordinate system. For example, taking a pixel (such as pixel a) with coordinates (xi, yi, zi) in the camera device coordinate system and (u, v) in the pixel coordinate system as an example, the coordinates of pixel a in the pixel coordinate system can be converted to the coordinates in the camera device coordinate system according to the following formula (2):

[0072]

[0073] In other words, based on the inverse matrix K of the intrinsic parameter matrix -1 This can convert the pixel coordinates of pixel a into the coordinates of pixel a in the camera device coordinate system.

[0074] Furthermore, based on the extrinsic parameters of the camera device, the coordinates of pixel a in the camera coordinate system are converted to coordinates in the vehicle coordinate system. Let the extrinsic parameters of the camera device be (x... c y c , z c θ xc θ yc θ zc For example, the extrinsic parameter matrix C of the camera device c It can be shown in the following formula (3):

[0075]

[0076] Where, x c y c , z c θ xc θ yc θ zc These represent the three-dimensional coordinates, yaw angle, pitch angle, and roll angle of the camera device in the vehicle coordinate system, respectively.

[0077] Taking the coordinates of a certain pixel a in the vehicle coordinate system as (Xi, Yi, Zi) as an example, the coordinates of pixel a in the camera coordinate system can be converted to the coordinates in the vehicle coordinate system according to the following formula (4):

[0078]

[0079] It should be noted that the origin o of the camera device coordinate system can be located at the center of the camera device, and the x, y, and z axes are defined by the camera device and follow the x, y, and z directions of the camera device base, respectively. The origin O of the vehicle coordinate system can be located at the projection point of the rear axle center of the vehicle body onto the ground, and the x, y, and z axes are the directions of the vehicle's front, left side, and vertically upward perpendicular to the vehicle's plane, respectively.

[0080] It is understandable that for each pixel in the image corresponding to a lane line, its coordinates in the vehicle coordinate system can be determined using formulas (2) and (4) above. Furthermore, by fitting the slope of each lane line in a frame of the image, the slope of each lane line in the vehicle coordinate system can be obtained. For example, Figure 5 The coordinate system shown can be considered as an example of the O-XY plane of the vehicle coordinate system, and the lines A to E can be respectively considered as Figure 4 The tangents to lane lines A through E in the vehicle coordinate system are such that each tangent is parallel to its corresponding lane line, or the point of tangency of each tangent lies on the Y-axis. It can be understood that the slope of each straight line from A to E can be considered as the slope of its corresponding lane line on the Y-axis of the vehicle coordinate system.

[0081] In some implementations, a vehicle can be determined to be traveling in a direction parallel to the lane lines when the slope of each lane line in each frame of an image is less than or equal to 0.1 (i.e., the angle between the lane line and the x-axis is less than or equal to 5°).

[0082] Figure 6 This illustration shows another exemplary flowchart of a sensor detection method provided in an embodiment of this application. The method 600 can be... Figure 2 The curb feature extraction module 222 and the result verification and statistics module 223 are executed as shown. More specifically, the method 600 includes S601 to S603, wherein S601 and S602 can be executed by the curb feature extraction module 222, and S603 can be executed by the result verification and statistics module 223.

[0083] S601 acquires point cloud data collected by lidar and extracts point cloud data corresponding to the road edge from the point cloud data.

[0084] Because road edges experience abrupt changes in geometric properties such as elevation and slope, these properties can be used to extract the corresponding point cloud data from a single frame of point cloud data. A single frame of point cloud data can be the data obtained by scanning with a LiDAR sensor rotating one full circle.

[0085] For example, suppose two adjacent point clouds (such as P) on a scan line in a frame of point cloud data. L-1 and P L The coordinates of ) in the lidar coordinate system are (x L-1 y L-1 , z L-1 ), (x L y L , z L The subscript L indicates the scanning order of the point cloud. The height difference between two adjacent points on a scanning line is then expressed as follows: Planar distance Spatial distance d L and the slope between adjacent points. L As shown in formulas (5) to (8):

[0086]

[0087] Furthermore, when the elevation difference is greater than or equal to the elevation difference threshold, the spatial distance is greater than or equal to the distance threshold, and the slope is greater than or equal to the included angle threshold, P can be... L Point cloud data corresponding to the road edge is marked and extracted. For example, the elevation difference threshold can be 0.08 meters, the distance threshold can be 0.05 meters, the included angle threshold can be 25°, or the aforementioned thresholds can be other values.

[0088] It is understood that the aforementioned method for extracting point cloud data corresponding to road edges is only an illustrative example. In actual implementation, other methods can also be used to extract point cloud data corresponding to road edges, such as using artificial intelligence algorithms.

[0089] S602 determines the slope of the road edge in the vehicle coordinate system based on the point cloud data corresponding to the road edge and the extrinsic parameters of the lidar.

[0090] For example, based on the extrinsic parameters of the LiDAR, the coordinates of each point cloud in the LiDAR coordinate system are converted to coordinates in the vehicle coordinate system, with the LiDAR extrinsic parameters as (x... L y L , z L θ xL θ yL θ zL For example, the extrinsic parameter matrix C of the lidar. L It can be shown in the following formula (9):

[0091]

[0092] Where, x L y L , z L θ xL θ yL θ zL These represent the three-dimensional coordinates, yaw angle, pitch angle, and roll angle of the lidar in the vehicle coordinate system, respectively.

[0093] Point cloud P L Taking the coordinates (XL, YL, ZL) in the vehicle coordinate system as an example, the point cloud P can be calculated according to the following formula (10). L Transforming coordinates from the lidar coordinate system to the vehicle coordinate system:

[0094]

[0095] It should be noted that the origin o of the lidar coordinate system can be located at the center of the lidar, and the x, y, and z axes are defined by the lidar and follow the x, y, and z directions of the lidar base, respectively. In some implementations, a height constraint z can be set to filter out stray points that do not meet the height constraint z from the data points corresponding to the road edge.

[0096] For example, after converting the point cloud corresponding to the curb into the vehicle coordinate system using the above formulas (9) and (10), the data corresponding to the curb within a preset range of the vehicle (e.g., a range of 5 to 30 meters in front of the vehicle) (the data after converting the point cloud data into the vehicle coordinate system) can be fitted to obtain the slope of each curb in the vehicle coordinate system of one or more curbs.

[0097] In some implementations, the coordinates are divided in the vehicle coordinate system as follows: Figure 7 The sampling grid shown allows for the selection of multiple data sets. Clustering and fitting of the data corresponding to the road edges in each set yields a slope value. More specifically, as... Figure 7 As shown in (a) and (b), the sampling grid is uniformly distributed on both sides of the X-axis of the vehicle coordinate system. The length of each sampling grid along the X-axis can be 5 meters, and the length along the Y-axis can be 9 meters, 10 meters, or other values. The distance between the grid closest to the Y-axis and the Y-axis is s. For example, s can be 6 meters, 7 meters, or other values. Further, for each sampling grid, the corresponding data within a circle with radius r centered on the grid center can be selected for clustering and fitting, where r can be 2 meters or 2.5 meters. It is understood that in actual implementation, the vehicle may not be traveling in a direction parallel to the lane lines, or the road edge may not be straight, so that not every grid can select data corresponding to the curb. For example, when the curb is curved, the position of the curb relative to the vehicle coordinate system can be as follows: Figure 7 As shown in (a), in this case, the data corresponding to the curb may only be selected in two grids. In this case, the curb slope determined based on the point cloud data of that frame is inaccurate, so the point cloud data of that frame can be discarded. When the data corresponding to the curb can be selected from multiple grids in the sampling grid, the data selected from each grid is clustered to obtain the data corresponding to the curb. Then, the slope of the curb portion corresponding to that grid can be obtained by fitting the clustered data corresponding to the curb. Furthermore, the samples selected from multiple grids (such as...) can be further... Figure 7The slopes obtained from the data selected from the five grids shown in (b) are sorted, and the standard deviations of multiple slopes are calculated after removing extreme values. When the standard deviation of the slope is less than or equal to 1.0, the curb of the current road segment can be considered straight or approximately straight, and the yaw angle of the lidar can be detected based on the curb of that road segment. For example, when the standard deviation of the slope is less than or equal to 1.0, the average of multiple slopes can be used as the slope of the curb of that road segment in the vehicle coordinate system.

[0098] In practical implementation, the size of the aforementioned sampling grid can vary in different scenarios. For example, the size of the sampling grid along the Y-axis can be adjusted according to the width of the road. Alternatively, data can be selected in the vehicle coordinate system using other methods in practical implementation.

[0099] S603 determines the angle by which the yaw angle of the lidar is offset based on the slope of the curb and the slope of the lane lines determined from the image.

[0100] For example, during the vehicle's travel in a direction parallel to the lane lines, the slope of the lane lines is determined based on the m-th frame image, and the slope of the road edge with reliable distribution stability is determined based on the n-th frame point cloud data. Then, the angle α between the lane lines and the X-axis of the vehicle coordinate system is determined based on the lane line slope, and the angle α' between the road edge and the X-axis of the vehicle coordinate system is determined based on the road edge slope. The difference between angle α and angle α' is the angle offset by the yaw angle of the lidar. The time difference between the starting time of the camera acquiring the m-th frame image and the starting time of the lidar acquiring the n-th frame point cloud data is less than or equal to a duration threshold. For example, this duration threshold can be 0.05 seconds, 0.04 seconds, or other values. Furthermore, whether the vehicle is traveling in a direction parallel to the lane lines can be determined according to method 300, which will not be elaborated here.

[0101] In some implementations, the result determined based on the aforementioned m-th frame image and n-th frame point cloud data can be considered as a single-frame detection result. For example, such as... Figure 7 As shown in (c), the position and slope of the lane line pixels in the vehicle coordinate system are indicated by the dashed line, and the position and slope of the curb point cloud in the vehicle coordinate system are indicated by the dashed line. The angle α between the solid line and the dashed line is the angle by which the yaw angle of the lidar is offset, as indicated by the single-frame detection result. Furthermore, the angle by which the yaw angle of the lidar is offset can be determined based on the average value of the single-frame detection results of 20 consecutive frames.

[0102] The sensor detection method provided in this application embodiment can detect the yaw angle of a lidar using images collected by a camera device. It has low computational resource consumption, good scene versatility, and since it is not limited by the initial value, it can detect a wider range of lidar yaw angles and has fast timeliness.

[0103] It is understood that the aforementioned methods 300 and 600 are illustrated using an image capture device to acquire a frontal image of the vehicle and a lidar to acquire point cloud data of the front of the vehicle as an example. In actual implementation, the yaw angle of the rear-facing lidar can also be verified using the sensor detection method provided in the embodiments of this application. When verifying the rear-facing lidar, an image of the front of the vehicle or an image of the rear of the vehicle can be used for verification. Furthermore, the aforementioned methods 300 and 600 are illustrated using the calculation of the slope of lane lines and curbs in the vehicle coordinate system as an example. In actual implementation, the pixel data corresponding to the lane lines and the point cloud data corresponding to the curbs can also be converted to the world coordinate system to determine the slope of the lane lines and curbs in the world coordinate system.

[0104] In some implementations, when the yaw angle of the lidar is determined to be greater than or equal to a preset threshold, the vehicle may perform any of the following actions: restrict intelligent driving functions or prompt the user to inspect the lidar. For example, the aforementioned intelligent driving functions may include, but are not limited to, the following functions affecting driving safety: automatic parking assist (APA), automatic valet parking (AVP), adaptive cruise control (ACC), lane cruise control (LCC), navigation cruise assist (NCA), forward collision warning, lane departure warning, lane keeping assist, and rear collision warning. The NCA function refers to the function of controlling the vehicle to travel towards the destination according to the navigation route and being able to control the vehicle to perform operations such as passing intersections, changing lanes, and shifting gears based on road information such as traffic lights.

[0105] Regarding the implementation method of prompting users to inspect and repair the lidar, Figure 8 This diagram illustrates a set of graphical user interfaces (GUIs) displayed on an in-vehicle display device. For example... Figure 8As shown, the vehicle's instrument panel displays three areas: 820, 830, and 840. Display area 820 can display the vehicle's current speed (e.g., 65 kilometers per hour (kph)), gear information (e.g., currently in Drive), minimum speed limit (e.g., 40 kph), maximum speed limit (e.g., 80 kph), remaining battery power (e.g., 80%), and remaining driving range (e.g., 460 kilometers). Display area 830 can display a virtual scene generated from data collected by sensors. This virtual scene includes the vehicle's icon and surrounding road environment information (e.g., information about other vehicles, lane markings, etc.). This virtual scene helps the user confirm the relative position of the vehicle to other vehicles. Display area 840 can display entertainment-related information, such as a music playback interface.

[0106] In some implementations, such as Figure 8 As shown in (a), when the yaw angle of the lidar is determined to be greater than or equal to a preset threshold, the vehicle can control the instrument panel to display a pop-up window 831, which includes the text "LiDAR malfunction, please check". In some other implementations, after determining that the yaw angle of the lidar is greater than or equal to the preset threshold, when a request to enable the LCC function is detected, the request is rejected and the instrument panel is controlled to display a pop-up window 832, which includes the text "LiDAR malfunction, lane keeping function cannot be enabled". In still other implementations, relevant prompts can also be played through a speaker or other sound-emitting device, such as playing the voice "LiDAR malfunction, please check" through the speaker at location 850. For example, a prompt can also be combined with a lighting device; for instance, when the yaw angle of the lidar is determined to be greater than or equal to the preset threshold, the ambient lighting inside the vehicle can be controlled to display a warning light such as a red light.

[0107] In actual implementation, if the yaw angle of the lidar is greater than or equal to a preset threshold, the virtual scene generated by the sensors, including the lidar, may be inaccurate. Therefore, the virtual scene may not be displayed in the display area 830.

[0108] It should be noted that the above Figure 8 The elements in the interface shown are for illustrative purposes only. In actual implementation, when the user is notified of an abnormality with the LiDAR via the in-vehicle display device, the interface displayed by the in-vehicle display device may also include elements similar to those shown. Figure 8 The different elements shown.

[0109] It should also be noted that the above combination Figures 3 to 8This section explains how to detect if the yaw angle of the lidar has deviated, and the vehicle's alerting and control methods when the yaw angle of the lidar deviates. In actual implementation, the method for detecting the yaw angle of the 4D millimeter-wave radar can be referred to the descriptions in methods 300 and 600 above. Specifically, the method for determining the curb slope based on the point cloud data of the 4D millimeter-wave radar, and the method for determining the angle of deviation of the 4D millimeter-wave radar's yaw angle based on the curb slope and lane line slope, can be referred to the description in method 600, and will not be repeated here.

[0110] Figure 9 A schematic flowchart of a sensor detection method provided in an embodiment of this application is shown. This method 900 can be applied to... Figure 1 In the vehicle shown, or the method can be derived from Figure 2 The system is executed as shown. More specifically, the method includes:

[0111] S910, acquire at least one frame of image captured by the first camera device within a first time period, and a group of point cloud data captured by the first radar within the first time period.

[0112] For example, the first camera device may include the camera device in method 300, and the first radar may be the lidar or 4D millimeter-wave radar in method 600. The method for acquiring at least one frame of image and point cloud data group refers to the description in method 300 and method 600, and will not be repeated here.

[0113] S920, based on at least one frame of image, determines the first slope of the lane line of the first road where the vehicle is located in the first coordinate system.

[0114] The first coordinate system can be either the vehicle coordinate system or the world coordinate system.

[0115] S930, determine the second slope of the curb of the first road in the first coordinate system based on the point cloud data set.

[0116] In some implementations, at least one image frame includes an M-th frame image, and the point cloud data set includes an N-th frame of point cloud data. The time difference between the start time of acquiring the M-th frame image and the start time of acquiring the N-th frame of point cloud data is less than or equal to a first threshold. Determining the first slope of the lane line of the first road where the vehicle is located in the first coordinate system based on at least one image frame includes: determining the first slope based on the M-th frame image. Determining the second slope of the curb of the first road in the first coordinate system based on the point cloud data set includes: determining the second slope based on the N-th frame of point cloud data.

[0117] For example, the first threshold and the aforementioned duration threshold can be the same value. For instance, the first threshold can be 0.05 seconds, or it can be any other duration. Furthermore, the specific implementation of determining the first slope based on a frame of image data can be found in the description of method 300, and the specific implementation of determining the second slope based on a frame of point cloud data can be found in the description of method 600, which will not be repeated here.

[0118] In actual implementation, multiple angle values ​​can be determined based on multiple sets of images and point cloud data. The difference in the acquisition time of each set of data in the multiple sets of images and point cloud data is less than or equal to a first threshold. Then, the first angle is determined based on the average value of multiple angle values. For a more specific implementation method, please refer to the description of method 600, which will not be repeated here.

[0119] In some implementations, a first set of data is determined based on the Nth frame of point cloud data. The first set of data indicates the position of the point cloud data corresponding to the road edge in the Nth frame of point cloud data in the first coordinate system. The first set of data is clustered and fitted to obtain multiple road edge data segments. The slope of each road edge data segment in the multiple road edge data segments is fitted to obtain multiple slope values. Each slope value in the multiple slope values ​​is a corresponding slope in the multiple road edge data segments. A second slope is determined based on the multiple slope values.

[0120] For example, each curb segment data can be selected from a sampling grid. More specifically, methods for determining a slope value and methods for determining a second slope can be referred to the description in method 600, which will not be repeated here.

[0121] S940, based on the first slope and the second slope, determines the first angle by which the yaw angle of the first radar deviates.

[0122] In some implementations, S940 can be further refined as follows: when the standard deviation corresponding to the aforementioned multiple slope values ​​is less than or equal to a preset threshold, the first angle is determined based on the first slope and the second slope.

[0123] For example, the preset threshold can be 1.0, or it can be other values. If the standard deviation of multiple slope values ​​is greater than the preset threshold, the second slope is discarded or ignored, that is, the second slope is no longer used to determine the angle of deviation of the yaw angle of the first radar.

[0124] In some implementations, at least one frame of image and point cloud data set is collected during the vehicle's travel in the first road. S940 can be further refined to: when the angle between the lane line and the vehicle's travel direction is less than or equal to a first angle threshold, determine the first angle based on the first slope and the second slope.

[0125] For example, the first included angle threshold can be 5°, or it can be 3°, or it can be other values.

[0126] In some implementations, the method further includes: when the first angle is greater than or equal to the second included angle threshold, the control prompting device prompts first information, the first information being used to indicate that the pose of the first radar is abnormal and / or to inspect the first radar.

[0127] For example, the second included angle threshold can be 10°, or it can be 5°, or it can be other values. The prompting device may include one or more of the vehicle-mounted display device, vehicle-mounted sound device, and vehicle-mounted lighting device in the foregoing embodiments, and the first information may include... Figure 8 The information in the pop-up window 831 shown may also include voice information played by the speaker at location 850.

[0128] In some implementations, the method further includes restricting the activation of intelligent driving functions when the first angle is greater than or equal to a second included angle threshold.

[0129] For example, the intelligent driving function may include one or more of the intelligent driving functions in the foregoing embodiments.

[0130] The sensor detection method provided in this application embodiment can quickly and accurately detect whether the yaw angle of the first radar in the vehicle has deviated from the calibration value. If it is determined that the yaw angle of the first radar has deviated too much, relevant information can be prompted and / or the intelligent driving function can be restricted, which helps to improve the safety of the vehicle.

[0131] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0132] The above text combines Figures 1 to 9 The methods provided in the embodiments of this application are described in detail below. Figure 10 and Figure 11 The apparatus provided in the embodiments of this application is described in detail. It should be understood that the description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content not described in detail, please refer to the method embodiments above. For the sake of brevity, it will not be repeated here.

[0133] Figure 10A schematic block diagram of a sensor detection device 2000 provided in an embodiment of this application is shown. The device 2000 may include units for executing the methods described in the foregoing embodiments. Furthermore, each unit in the device 2000 implements a corresponding process of the above-described method embodiments. The device 2000 includes an acquisition unit 2010, which can be used to implement corresponding data acquisition or transmission / reception functions. The device 2000 also includes a processing unit 2020, which can be used to implement corresponding processing functions.

[0134] Optionally, the device 2000 further includes a storage unit, which can be used to store instructions and / or data. The processing unit 2020 can read the instructions and / or data in the storage unit so that the device can perform the relevant actions in the aforementioned method embodiments.

[0135] It should be understood that the specific process of each unit performing the above-mentioned corresponding steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0136] It should also be understood that the device 2000 described herein is embodied in the form of a functional unit. The terms “module” or “unit” may refer to application-specific ASICs, electronic circuits, processors (e.g., shared processors, proprietary processors, or group processors) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.

[0137] The apparatus in this embodiment has the function of implementing the corresponding steps in the aforementioned method. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the acquisition unit 2010 can be replaced by a transceiver, and other units, such as the processing unit, can be replaced by a processor, used to execute the relevant processing operations in each method embodiment.

[0138] For example, the acquisition unit 2010 and the processing unit 2020 can be set in Figure 1 In the vehicle 100 shown, or it can also be set in Figure 2 In the illustrated system, more specifically, the acquisition unit 2010 and processing unit 2020 can be housed within the detection module 220. Exemplarily, the operations performed by the acquisition unit 2010 and processing unit 2020 can be executed by a single processor, or by different processors. In specific implementations, the one or more processors can be configured as follows: Figure 1 The processor in the vehicle 100 shown; or, the device 2000 described above may be a chip disposed in the vehicle 100.

[0139] In the specific implementation process, the units in the above device can be fully or partially integrated together, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SoC).

[0140] Figure 11 This is another schematic block diagram of the sensor detection device provided in the embodiments of this application. Figure 11 The illustrated device 2100 may include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, transceiver 2120, and memory 2130 are connected via internal interconnects. The memory 2130 stores instructions, and the processor 2110 executes the instructions stored in the memory 2130 to implement the methods described in the above embodiments. Optionally, the memory 2130 may be coupled to the processor 2110 via an interface or integrated with the processor 2110.

[0141] It should be noted that the transceiver 2120 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to realize communication between device 2100 and other devices or communication networks.

[0142] Memory 2130 can be volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes various forms such as: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0143] Transceiver 2120 uses transceiver devices, such as but not limited to transceivers, to enable communication between device 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.

[0144] This application also provides a vehicle that includes the device 2000 or device 2100 described in the above embodiments.

[0145] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to implement the methods described in the above embodiments of this application.

[0146] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to implement the methods described in the above embodiments of this application.

[0147] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments of this application.

[0148] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0149] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0150] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0152] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A sensor detection method, characterized in that, include: The vehicle includes the first camera device and the first radar. At least one frame of image captured by the first camera device within a first time period and a point cloud data set captured by the first radar within the first time period are acquired. The first slope of the lane line of the first road where the vehicle is located is determined in a first coordinate system based on the at least one frame of image, wherein the first coordinate system is a vehicle coordinate system or a world coordinate system; The second slope of the curb of the first road in the first coordinate system is determined based on the point cloud data set. Based on the first slope and the second slope, the first angle by which the yaw angle of the first radar is offset is determined.

2. The method according to claim 1, characterized in that, The at least one frame image includes the Mth frame image, the point cloud data group includes the Nth frame point cloud data, and the time difference between the start time of acquiring the Mth frame image and the start time of acquiring the Nth frame point cloud data is less than or equal to a first threshold. Determining the first slope of the lane line of the first road where the vehicle is located in the first coordinate system based on the at least one frame of image includes: The first slope is determined based on the Mth frame image; Determining the second slope of the curb of the first road in the first coordinate system based on the point cloud data set includes: The second slope is determined based on the point cloud data of the Nth frame.

3. The method according to claim 2, characterized in that, Determining the second slope based on the point cloud data of the Nth frame includes: A first set of data is determined based on the Nth frame point cloud data, and the first set of data indicates the position of the point cloud data corresponding to the curb in the Nth frame point cloud data in the first coordinate system; Clustering and fitting of the first set of data yields multiple road edge data segments; Slope fitting is performed on each road edge data segment in the multiple road edge data segments to obtain multiple slope values, where each slope value is a corresponding slope in the multiple road edge data segments; The second slope is determined based on the plurality of slope values.

4. The method according to claim 3, characterized in that, The step of determining the first angle offset by the yaw angle of the first radar based on the first slope and the second slope includes: When the standard deviation corresponding to the plurality of slope values ​​is less than or equal to a preset threshold, the first angle is determined based on the first slope and the second slope.

5. The method according to any one of claims 1 to 4, characterized in that, The at least one frame image and the point cloud data set were collected during the vehicle's travel on the first road. Determining the first angle offset by the yaw angle of the first radar based on the first slope and the second slope includes: When the first slope indicates that the angle between the lane line and the vehicle's direction of travel is less than or equal to a first angle threshold, the first angle is determined based on the first slope and the second slope.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the first angle is greater than or equal to the second included angle threshold, the control prompting device prompts the first information, which is used to indicate that the pose of the first radar is abnormal and / or to repair the first radar.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: When the first angle is greater than or equal to the second included angle threshold, the activation of intelligent driving functions is restricted.

8. A sensor detection device, characterized in that, include: The acquisition unit is used to acquire at least one frame of image captured by the first camera device within a first time period, and a point cloud data group captured by the first radar within the first time period. The vehicle includes the first camera device and the first radar. Processing unit, used for: The first slope of the lane line of the first road where the vehicle is located is determined in a first coordinate system based on the at least one frame of image, wherein the first coordinate system includes a vehicle coordinate system or a world coordinate system; The second slope of the curb of the first road in the first coordinate system is determined based on the point cloud data set. Based on the first slope and the second slope, the first angle by which the yaw angle of the first radar is offset is determined.

9. The apparatus according to claim 8, characterized in that, The at least one frame image includes the Mth frame image, the point cloud data group includes the Nth frame point cloud data, and the time difference between the start time of acquiring the Mth frame image and the start time of acquiring the Nth frame point cloud data is less than or equal to a first threshold. The processing unit is used for: The first slope is determined based on the Mth frame image; The second slope is determined based on the point cloud data of the Nth frame.

10. The apparatus according to claim 9, characterized in that, The processing unit is used for: A first set of data is determined based on the Nth frame point cloud data, and the first set of data indicates the position of the point cloud data corresponding to the curb in the Nth frame point cloud data in the first coordinate system; Clustering and fitting of the first set of data yields multiple road edge data segments; Slope fitting is performed on each road edge data segment in the multiple road edge data segments to obtain multiple slope values, where each slope value is a corresponding slope in the multiple road edge data segments; The second slope is determined based on the plurality of slope values.

11. The apparatus according to claim 10, characterized in that, The processing unit is used for: When the standard deviation corresponding to the plurality of slope values ​​is less than or equal to a preset threshold, the first angle is determined based on the first slope and the second slope.

12. The apparatus according to any one of claims 8 to 11, characterized in that, The at least one frame image and the point cloud data set were collected during the vehicle's travel on the first road, and the processing unit is used for: When the first slope indicates that the angle between the lane line and the vehicle's direction of travel is less than or equal to a first angle threshold, the first angle is determined based on the first slope and the second slope.

13. The apparatus according to any one of claims 8 to 12, characterized in that, The processing unit is also used for: When the first angle is greater than or equal to the second included angle threshold, the control prompting device prompts the first information, which is used to indicate that the pose of the first radar is abnormal and / or to repair the first radar.

14. The apparatus according to any one of claims 8 to 13, characterized in that, The processing unit is also used for: When the first angle is greater than or equal to the second included angle threshold, the activation of intelligent driving functions is restricted.

15. A sensor detection device, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 7.

16. The apparatus according to claim 15, characterized in that, The device also includes the memory.

17. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

18. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 7.

19. A computer program product, characterized in that, The computer program product includes: computer program code, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.

20. A vehicle, characterized in that, Includes the apparatus as described in any one of claims 8 to 16, or the computer-readable storage medium as described in claim 17, or the chip as described in claim 18, or the vehicle is equipped with the computer program product as described in claim 19.

Images