Sensor detection method and apparatus, and vehicle

Abstract

A sensor detection method and apparatus, and a vehicle. The method comprises: acquiring at least one image frame collected by a first camera apparatus within a first time period and a point cloud data set collected by a first radar within the first time period; on the basis of the at least one image frame, determining a first slope of a lane line of a first road where a vehicle is located in a vehicle coordinate system, wherein the vehicle comprises the first camera apparatus and the first radar; on the basis of the point cloud data set, determining a second slope of a road edge of the first road in the vehicle coordinate system; and on the basis of the first slope and the second slope, determining a first angle by which a yaw angle of the first radar is offset. The technical solution can be applied to intelligent vehicles such as electric vehicles and new-energy vehicles, and can quickly and accurately detect whether a yaw angle of a first radar in a vehicle is offset relative to a calibration value. When a first angle is greater than or equal to a preset threshold value, a user can be prompted to perform maintenance and / or restrict the use of an intelligent driving function of the vehicle, thereby helping to improve the safety of the vehicle.

Description

Sensor detection method, device and vehicle TECHNICAL FIELD

[0001] The present application relates to the field of intelligent driving, and more particularly, to a sensor detection method, device and vehicle. BACKGROUND

[0002] With the increasing intelligence of vehicles, the accuracy and reliability of the sensing results of sensors are increasingly required. In the field of intelligent driving, radar sensors (such as laser radars) are key sensors for real-time sensing of surrounding elements (such as roads, lanes, traffic signal lights, signs, etc.), and high-precision calibration of the radar sensors is a prerequisite for achieving high-precision sensing functions.

[0003] However, in the actual use of vehicles, if a large deviation occurs between the actual yaw angle of the radar sensor and the yaw angle calibrated at the vehicle end due to external forces such as collision, the sensing results of the radar sensor will be inaccurate. In the process of controlling the driving of the vehicle based on the sensing results of the radar sensor, inaccurate sensing results of the radar sensor can cause the vehicle to deviate from the intended direction, or cause the vehicle to trigger an autonomous emergency braking (AEB) and other sudden braking, resulting in safety risks of the vehicle.

[0004] Therefore, a sensor detection scheme capable of effectively detecting whether the yaw angle of the radar sensor is accurate is urgently needed.

[0005] SUMMARY

[0006] The present application provides a sensor detection method, device and vehicle, which can quickly and accurately detect whether the yaw angle of the radar in the vehicle deviates from the calibrated value, thereby helping to improve the safety of the vehicle.

[0007] In a first aspect, a sensor detection method is provided, which can be executed by a vehicle, for example, can be executed by a computing platform of the vehicle, or can also be executed by a chip or circuit for the vehicle.

[0008] The method comprises: acquiring at least one frame of image collected by a first camera device in a first time period and point cloud data set collected by a first radar in the first time period, the vehicle comprising the first camera device and the first radar; determining a first slope of a lane line of a first road where the vehicle is located in a first coordinate system according to the at least one frame of image, the first coordinate system being a vehicle coordinate system or a world coordinate system; determining a second slope of a curb of the first road in the first coordinate system according to the point cloud data set; and determining a first angle deviated by a yaw angle of the first radar according to the first slope and the second slope.

[0009] In some implementations, the first radar is a laser radar; or, the first radar is a millimeter wave radar, such as a four-dimensional (4D) millimeter wave radar; or, the first radar can also be other radars with height measurement capability.

[0010] In the technical solution described above, the yaw angle of the first radar is verified by using the image collected by the camera device, the calculation resource consumption is low, and the scene versatility is good. Since it is not limited to the initial value, the coverage range of the yaw angle of the first radar that can be detected is larger, and the timeliness is fast.

[0011] With reference to the first aspect, in some implementations of the first aspect, the at least one frame of image includes an Mth frame of image, the group of point cloud data includes an Nth frame of point cloud data, and a time difference between a starting time of collecting the Mth frame of image and a starting time of collecting the Nth frame of point cloud data is less than or equal to a first threshold value; determining, according to the at least one frame of image, a first slope of a lane line of a first road on which the vehicle is located in the first coordinate system, includes: determining the first slope according to the Mth frame of image; determining, according to the group of point cloud data, a second slope of a curb of the first road in the first coordinate system, includes: determining the second slope according to the Nth frame of point cloud data.

[0012] With reference to the first aspect, in some implementations of the first aspect, determining the second slope according to the Nth frame of point cloud data includes: determining a first group of data according to the Nth frame of point cloud data, the first group of data indicating positions of point cloud data corresponding to the curb in the first coordinate system in the Nth frame of point cloud data; clustering and fitting the first group of data to obtain a plurality of pieces of curb data; performing slope fitting on each piece of curb data in the plurality of pieces of curb data to obtain a plurality of slope values, each slope value in the plurality of slope values being a corresponding slope in the plurality of pieces of curb data; and determining the second slope according to the plurality of slope values.

[0013] Since in actual implementation, the curb of the road can not be straight, by segmenting, clustering and fitting the data corresponding to the curb (or road edge) to obtain a plurality of slope values, the slopes of the curbs at different positions in the road can be more accurately reflected, and then the slope of the curb is determined according to the plurality of slope values, which helps to improve the reliability of the determined slope of the curb, thereby improving the accuracy of the determined angle by which the yaw angle of the first radar deviates.

[0014] With reference to the first aspect, in some implementations of the first aspect, determining, according to the first slope and the second slope, a first angle by which the yaw angle of the first radar deviates, includes: when a standard deviation corresponding to the plurality of slope values is less than or equal to a preset threshold value, determining the first angle according to the first slope and the second slope.

[0015] In the technical solution, when the plurality of slope values are close to each other, it can be determined that the curb of the first road is flat or approximately flat, and in this case, the accuracy of the offset angle of the yaw angle of the first radar determined according to the slope of the curb and the slope of the lane line is more reliable.

[0016] With reference to the first aspect, in some implementations of the first aspect, the at least one frame of image and the point cloud data set are collected during driving of the vehicle on the first road, and the first angle offset by the yaw angle of the first radar is determined according to the first slope and the second slope, including: when the first slope indicates that the included angle between the lane line and the driving direction of the vehicle is less than or equal to the first included angle threshold, the first angle is determined according to the first slope and the second slope.

[0017] In the technical solution, when the included angle between the lane line and the driving direction of the vehicle is less than or equal to a certain threshold, the vehicle can be considered to be driving in a direction parallel to the lane, in which case the calculation complexity required to determine the slope of the curb is lower, and the reliability of the angle offset by the yaw angle of the first radar based on the slope of the curb and the slope of the lane line is higher.

[0018] With reference to the first aspect, in some implementations of the first aspect, the method further includes: when the first angle is greater than or equal to a second included angle threshold, controlling the prompt device to prompt first information, the first information being used to prompt that the pose of the first radar is abnormal and / or to maintain the first radar.

[0019] In the technical solution, when the yaw angle of the first radar is offset by too large an angle, prompting the first information helps to prompt the user to maintain the first radar to eliminate the abnormality as soon as possible.

[0020] With reference to the first aspect, in some implementations of the first aspect, the method further includes: when the first angle is greater than or equal to a second included angle threshold, limiting the enabling of the intelligent driving function.

[0021] In the case where the yaw angle of the first radar is offset by too much, if the intelligent driving function is enabled, it can cause the vehicle to make an unintended turn, or cause the AEB to be triggered to make an emergency stop, etc., so that the vehicle has a safety risk. Therefore, in the technical solution, the enabling of the intelligent driving function is limited, which helps to improve the safety of the vehicle.

[0022] In a second aspect, a sensor detection apparatus is provided, which comprises an acquisition unit and a processing unit, wherein the acquisition unit is configured to: acquire at least one image frame collected by a first camera in a first time period and a point cloud data set collected by a first radar in the first time period, the vehicle comprising the first camera and the first radar; and the processing unit is configured to: determine a first slope of a lane line of a first road in which the vehicle is located in a first coordinate system according to the at least one image frame, the first coordinate system comprising a vehicle coordinate system or a world coordinate system; determine a second slope of a curb of the first road in the first coordinate system according to the point cloud data set; and determine a first angle by which a yaw angle of the first radar deviates according to the first slope and the second slope.

[0023] With reference to the second aspect, in some implementations of the second aspect, the at least one image frame comprises an Mth image frame, and the point cloud data set comprises an Nth point cloud data, a time difference between a starting time of collecting the Mth image frame and a starting time of collecting the Nth point cloud data is less than or equal to a first threshold value; and the processing unit is configured to: determine the first slope according to the Mth image frame; and determine the second slope according to the Nth point cloud data.

[0024] With reference to the second aspect, in some implementations of the second aspect, the processing unit is configured to: determine a first group of data according to the Nth point cloud data, the first group of data indicating positions of point cloud data corresponding to the curb in the first coordinate system in the Nth point cloud data; perform clustering and fitting on the first group of data to obtain a plurality of curb data segments; perform slope fitting on each of the plurality of curb data segments to obtain a plurality of slope values, each of the plurality of slope values being a corresponding slope of one of the plurality of curb data segments; and determine the second slope according to the plurality of slope values.

[0025] With reference to the second aspect, in some implementations of the second aspect, the processing unit is configured to: when a standard deviation corresponding to the plurality of slope values is less than or equal to a preset threshold value, determine the first angle according to the first slope and the second slope.

[0026] With reference to the second aspect, in some implementations of the second aspect, the at least one image frame and the point cloud data set are collected in a process in which the vehicle travels on the first road, and the processing unit is configured to: when the first slope indicates that an included angle between the lane line and a traveling direction of the vehicle is less than or equal to a first included angle threshold value, determine the first angle according to the first slope and the second slope.

[0027] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to: when the first angle is greater than or equal to a second included angle threshold value, control a prompt apparatus to prompt first information, the first information being used to prompt that a pose of the first radar is abnormal and / or to maintain the first radar.

[0028] With reference to the second aspect, in some implementations of the second aspect, the processing unit is further configured to: limit the enabling of the intelligent driving function when the first angle is greater than or equal to a second included angle threshold.

[0029] In a third aspect, a sensor detection apparatus is provided, the apparatus comprising: a processor configured to execute a computer program stored in the memory to cause the apparatus to perform the method of any possible implementation of the first aspect.

[0030] With reference to the third aspect, in some implementations of the third aspect, the apparatus further comprises a memory.

[0031] In a fourth aspect, a computer program product is provided, the computer program product comprising: computer program code which, when executed on a computer or processor, causes the computer or processor to perform the method of any possible implementation of the first aspect.

[0032] It should be noted that the computer program code can be stored in whole or in part on a storage medium, which can be packaged together with the processor or separately from the processor.

[0033] In a fifth aspect, a computer-readable storage medium is provided, the computer-readable medium storing instructions which, when executed on a processor, cause the processor to implement the method of any possible implementation of the first aspect.

[0034] In a sixth aspect, a chip is provided, the chip comprising circuitry configured to perform the method of any possible implementation of the first aspect.

[0035] In a seventh aspect, a vehicle is provided, the vehicle comprising the apparatus of any possible implementation of the second aspect or the third aspect, or the vehicle comprising the computer-readable storage of any possible implementation of the fifth aspect, or the vehicle comprising the chip of any possible implementation of the sixth aspect, or the vehicle being loaded with the computer program code of any possible implementation of the fourth aspect.

[0036] With reference to the seventh aspect, in some implementations of the seventh aspect, the vehicle is a vehicle in a broad sense, for example, can be a traffic tool (such as a commercial vehicle, a passenger vehicle, a motorcycle, a flying vehicle, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural device (such as a mower, a harvester, etc.), a recreational device, a toy vehicle, etc. In actual implementation, the vehicle can also be a road traffic tool, a water traffic tool, an air traffic tool, an industrial device, an agricultural device, or an entertainment device, etc. other intelligent driving devices.

[0037] The beneficial effects not described in the second aspect to the seventh aspect can be referred to the description in the first aspect, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0038] FIG. 1 is a functional schematic block diagram of a vehicle according to an embodiment of the present application;

[0039] FIG. 2 is a schematic block diagram of a sensor detection system architecture according to an embodiment of the present application;

[0040] FIG. 3 is a schematic flow chart of a sensor detection method according to an embodiment of the present application;

[0041] FIG. 4 is a schematic diagram of extracting lane line features from images captured by a camera according to an embodiment of the present application;

[0042] FIG. 5 is a schematic diagram of the position of a lane line in a vehicle coordinate system according to an embodiment of the present application;

[0043] FIG. 6 is another schematic flow chart of a sensor detection method according to an embodiment of the present application;

[0044] FIG. 7 is a schematic diagram of a grid required for data selection and detection results according to an embodiment of the present application;

[0045] FIG. 8 is a schematic diagram of an application scenario of a sensor detection method according to an embodiment of the present application;

[0046] FIG. 9 is a schematic flow chart of a sensor detection method according to an embodiment of the present application;

[0047] FIG. 10 is a schematic block diagram of a sensor detection apparatus according to an embodiment of the present application;

[0048] FIG. 11 is another schematic block diagram of a sensor detection apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0050] FIG. 1 is a functional block diagram of a vehicle according to an embodiment of the present application. As shown in FIG. 1, the vehicle 100 can include a perception system 120 and a computing platform 150, wherein the perception system 120 can include several sensors for sensing information of an environment around the vehicle 100. For example, the perception system 120 can include a positioning system, which can be a global navigation satellite system (GNSS, such as a global positioning system (GPS), a Beidou system, etc.). For another example, the perception system 120 can further include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0051] Some or all of the functionality of the vehicle 100 can be controlled by the computing platform 150. The computing platform 150 can include processors 151-15n, which are circuits that have the capability to process signals. In one implementation, the processors can be circuits that have the capability to read and execute instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a type of microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processors can be circuits that implement functionality through fixed or reconfigurable logic, such as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) such as a field programmable gate array (FPGA). In reconfigurable hardware circuits, the processor loads configuration files to implement the configuration of the hardware circuit, which can be understood as the processor loading instructions to implement the corresponding functionality. Additionally, the processors can be hardware circuits designed for artificial intelligence, which can be understood as a type of ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like. Additionally, the computing platform 150 can include a memory that stores instructions that can be called by some or all of the processors 151-15n to implement corresponding functionality.

[0052] The operation of the intelligent driving system can be controlled by the computing platform 150, which can include an advanced driving assistant system (ADAS) and an autonomous driving system (ADS). The intelligent driving system uses various sensors on the vehicle (including but not limited to: lidar, millimeter wave radar, camera, ultrasonic sensor, global positioning system, inertial measurement unit) to obtain information from the surroundings of the vehicle, and analyzes and processes the obtained information to realize functions such as obstacle perception, target recognition, vehicle positioning, path planning, driver monitoring / alerting, etc., thereby improving the safety, automation level and comfort of vehicle driving.

[0053] At different levels of autonomous driving (or intelligent driving, a total of L0-L5 six levels), based on artificial intelligence algorithms and information obtained by multiple sensors, the intelligent driving system can realize different levels of autonomous driving assistance. The above-mentioned autonomous driving levels are based on the classification standard of the Society of Automotive Engineers (SAE). Among them, L0 level is no automation; L1 level is driving assistance; L2 level is partial automation; L3 level is conditional automation; L4 level is high automation; L5 level is full automation. The tasks of monitoring the road conditions and reacting are completed by the driver and the system together at L1 to L3 levels, and the driver needs to take over the dynamic driving task. L4 and L5 levels can make the driver completely change to the role of a passenger. At present, the functions that the intelligent driving system can realize mainly include but are not limited to: adaptive cruise assistance, automatic emergency braking, automatic parking, blind spot monitoring, front intersection traffic warning / braking, rear intersection traffic warning / braking, front vehicle collision warning, lane departure warning, lane keeping assistance, rear vehicle collision warning, traffic sign recognition, traffic congestion assistance, highway assistance, etc. It should be understood that the above-mentioned various functions can have specific modes at different autonomous driving levels (L0-L5), and the higher the autonomous driving level, the more intelligent the corresponding mode.

[0054] In this application, when the image collected by the camera in the perception system 120 includes lane line information, the computing platform 150 can determine the slope of the lane line in the vehicle coordinate system according to the image; in addition, the computing platform 150 can also extract the curb feature according to the point cloud data collected by the lidar in the perception system 120, and determine the slope of the curb according to the curb feature. Further, the computing platform 150 can determine whether the yaw angle of the lidar is consistent with the calibration value according to the slope of the lane line and the slope of the curb.

[0055] FIG. 2 shows a schematic block diagram of a sensor detection system architecture according to an embodiment of the present application. The system includes a perception module 210, a detection module 220, and a control module 230. In some implementations, the system further includes a prompting module 240. Specifically,

[0056] The perception module 210 can include one or more cameras in the perception system 120 shown in FIG. 1, and one or more lidars, for collecting environmental information of an area where the vehicle is located, such as information of parking lines, information of road edges, etc. The perception module 210 can send the images and point cloud data collected by it to the detection module 220.

[0057] The detection module 220 can include one or more processors in the computing platform 150 shown in FIG. 1, for detecting whether the yaw angle of the lidar deviates. The detection module 220 includes a lane line detection module 221, a curb feature extraction module 222, and a result verification and statistics module 223. The lane line detection module 221 is configured to determine the slope of the lane line in the road relative to the vehicle coordinate system according to the images collected by the camera. The curb feature extraction module 222 is configured to determine the slope of the edge of the road relative to the vehicle coordinate system according to the point cloud data collected by the lidar or the 4D millimeter wave radar. The result verification and statistics module 223 is configured to determine the difference between the actual yaw angle and the calibrated yaw angle of the lidar or the 4D millimeter wave radar according to the statistical value of multiple frames of detection results.

[0058] The control module 230 can include one or more processors in the computing platform 150 shown in FIG. 1, for limiting the use of intelligent driving functions when the difference between the actual yaw angle and the calibrated yaw angle of the lidar is greater than or equal to a difference threshold value.

[0059] The prompt module 240 can include one or more of a vehicle-mounted display device, a vehicle-mounted sound device (such as a speaker, a sound system, etc.), a vehicle-mounted light device (such as an atmosphere lamp associated with a sensor, etc.), for prompting that the laser radar or 4D millimeter wave radar is abnormal when the difference between the actual yaw angle and the calibrated yaw angle of the laser radar or 4D millimeter wave radar is greater than or equal to the difference threshold value, so that the user can timely maintain the laser radar or 4D millimeter wave radar. Exemplarily, the vehicle-mounted display device can include a vehicle-mounted display screen and a projection display screen. The vehicle-mounted display screen is a physical display screen and is an important part of the vehicle information entertainment system. Multiple display screens can be arranged in the cabin, such as a digital instrument display screen, a central control screen, a display screen in front of a passenger (also referred to as a front passenger) at a co-driver position, a display screen in front of a left rear passenger, and a display screen in front of a right rear passenger, or even a vehicle window can be used as a display screen for display. The projection display screen can include a head-up display, also known as a head-up display system. It is mainly used for displaying driving information such as speed, navigation, etc. on the display device (such as the windshield) in front of the driver. In order to reduce the driver's visual transfer time and avoid pupil changes caused by the driver's visual transfer, and to improve driving safety and comfort. The HUD includes, for example, a combiner-HUD (C-HUD) system, a windshield-HUD (W-HUD) system, and an augmented reality HUD (AR-HUD).

[0060] It should be understood that the above module is only an example, and in actual application, the above module can be added or deleted according to actual needs. For example, in the system architecture shown in FIG. 2, the prompt module 240 and the detection module 220 are combined into one module.

[0061] The above describes the system related to the embodiments of the present application in combination with FIG. 2, and the sensor detection method based on the system shown in FIG. 2 is described in detail below.

[0062] FIG. 3 shows an exemplary flowchart of a sensor detection method provided by the embodiments of the present application. The method 300 can be performed by the lane line detection module 221 shown in FIG. 2, and the method 300 includes the following steps.

[0063] S301, acquiring an image collected by a camera device, and extracting pixel points corresponding to a plurality of lane lines from the image.

[0064] Exemplarily, the camera device can be a camera device arranged outside the vehicle cabin, for example, can include any one of a front-view camera device, a rear-view camera device, a surround-view camera device for acquiring a vehicle front-side image or for acquiring a vehicle rear-side image. Among them, the front-view camera device can be installed at the front windshield; the rear-view camera device can be installed at the rear trunk; the surround-view camera device includes four camera devices installed around the vehicle, and the images acquired by the four camera devices can be spliced to obtain a panoramic image of the vehicle around.

[0065] Exemplarily, in the process of vehicle driving, the camera device collects a video stream, and the aforementioned image can include a frame image in the video stream.

[0066] S302, according to the pixel points corresponding to the lane lines and the external parameters of the camera device, the position and slope of each lane line in the vehicle coordinate system are determined.

[0067] Exemplarily, a frame image collected by the front-view camera device of the vehicle can be as shown in (a) of FIG. 4, which includes the pixels of lane lines 401, 402 and 403, and the pixels of kerbs 404 and 405, in addition to the pixels of other vehicles. Further, the pixels related to the lane lines are extracted by an image recognition algorithm, and exemplarily, lines A to E shown in (b) of FIG. 4 are respectively regarded as the pixels corresponding to lane lines 401 to 403, and the pixels corresponding to kerbs 404 and 405 in the image.

[0068] Further, according to the external parameters of the camera device and the internal parameters of the camera device, the position of each pixel point in the image in the vehicle coordinate system can be determined. It can be understood that the internal parameters are the parameters of the camera device itself, for example, focal length, etc. The external parameters are the parameters related to the installation position of the camera device, for example, pitch, roll and yaw, etc.

[0069] Exemplarily, the internal parameter matrix K of the camera device can be as shown in the following formula (1):

[0070] wherein f x may represent the length of the focal length in the x-axis direction of the image coordinate system described using pixels, f y may represent the length of the focal length in the y-axis direction of the image coordinate system described using pixels, p x and p y may be used to represent the position of a pixel point in the image coordinate system, with the unit of pixel.

[0071] According to the intrinsic matrix of the camera, the coordinates in the pixel coordinate system can be converted into the coordinates in the camera coordinate system. For example, taking the coordinates of a certain pixel point (such as pixel point a) in the camera coordinate system as (xi, yi, zi) and the coordinates in the pixel coordinate system as (u, v) as an example, the coordinates of the pixel point a in the pixel coordinate system can be converted into the coordinates in the camera coordinate system according to the following formula (2):

[0072] That is, according to the inverse matrix K -1 of the intrinsic matrix, the pixel coordinates of the pixel point a can be converted into the coordinates of the pixel point a in the camera coordinate system.

[0073] Further, according to the extrinsic parameters of the camera, the coordinates of the pixel point a in the camera coordinate system are converted into the coordinates in the vehicle coordinate system. Taking the extrinsic parameters of the camera as (x c , y c , z c , θ xc , θ yc , θ zc ) as an example, the extrinsic matrix C c of the camera can be as shown in the following formula (3):

[0074] Wherein, x c , y c , z c , θ xc , θ yc , θ zc respectively represent the three-dimensional coordinates, the yaw angle, the pitch angle and the roll angle of the camera in the vehicle coordinate system.

[0075] Taking the coordinates of a certain pixel point a in the vehicle coordinate system as (Xi, Yi, Zi) as an example, the coordinates of the pixel point a in the camera coordinate system can be converted into the coordinates in the vehicle coordinate system according to the following formula (4):

[0076] It should be noted that the origin o of the camera coordinate system can be located at the center of the camera, and the x, y and z axes are respectively defined by the camera to follow the x, y and z directions of the camera base. The origin O of the vehicle coordinate system can be located at the projection point of the center of the rear axle of the vehicle body on the ground, and the X, Y and Z axes are respectively the front direction of the vehicle body, the left direction of the vehicle body and the direction perpendicular to the vehicle body plane vertically upward.

[0077] It can be understood that for each pixel point corresponding to a lane line in the image, the coordinates of the pixel point in the vehicle coordinate system can be determined respectively through the above formula (2) and formula (4). Further, the slope of each lane line in the vehicle coordinate system can be obtained by performing linear slope fitting on each lane line in an image. Exemplarily, the coordinate system shown in FIG. 5 can be regarded as an example of the O-XY plane of the vehicle coordinate system, and the straight lines A to E can be regarded as the tangents corresponding to the lane lines A to E in FIG. 4 respectively, each tangent is parallel to the corresponding lane line, or the tangent point of each tangent is located on the Y axis. It can be understood that the slope of each straight line in the straight lines A to E can be regarded as the slope of the corresponding lane line at the Y axis of the vehicle coordinate system.

[0078] In some implementations, when the slope of each lane line in all lane lines in each image is less than or equal to 0.1 (that is, the angle between the lane line and the x axis is less than or equal to 5°), it can be determined that the vehicle is currently driving in a direction parallel to the lane line.

[0079] FIG. 6 shows another exemplary flowchart of a sensor detection method provided by the embodiments of the present application, the method 600 can be executed by the curb feature extraction module 222 and the result verification and statistics module 223 shown in FIG. 2, more specifically, the method 600 includes S601 to S603, wherein S601 and S602 can be executed by the curb feature extraction module 222, and S603 can be executed by the result verification and statistics module 223. Wherein:

[0080] S601, acquiring point cloud data collected by a laser radar, and extracting point cloud data corresponding to a curb from the point cloud data.

[0081] Since the elevation, slope and other geometric properties of the curb of the road will change abruptly, for a frame of point cloud data, these properties can be used to extract point cloud data corresponding to the curb from the frame of point cloud data. Wherein, the frame of point cloud data can be data obtained by scanning one revolution of the motor of the laser radar.

[0082] Exemplarily, assuming that the coordinates of two adjacent point clouds (such as P L-1 and P L ) on a scanning line in a frame of point cloud data in the laser radar coordinate system are (x L-1 , y L-1 , z L-1 ) and (x L , y L , z L ), and the subscript L represents the scanning order of the point cloud, then the height difference between the points corresponding to the two adjacent point clouds on the scanning line is the planar distance is the spatial distance is and the slope s between adjacent points L As shown in formulas (5) to (8):

[0083] Further, when the height difference is greater than or equal to a height difference threshold, the spatial distance is greater than or equal to a distance threshold, and the slope is greater than or equal to an angle threshold, the point cloud data corresponding to the curb can be determined as P L The point cloud data corresponding to the curb is marked and extracted. Exemplarily, the height difference threshold can be 0.08 meters, the distance threshold can be 0.05 meters, and the angle threshold can be 25°, or the foregoing thresholds can also be other numerical values.

[0084] It can be understood that the foregoing method of extracting point cloud data corresponding to the curb is only an exemplary description, and in actual implementation, the point cloud data corresponding to the curb can also be extracted by other methods, such as by an artificial intelligence algorithm.

[0085] S602, according to the point cloud data corresponding to the curb and the extrinsic parameters of the lidar, determining the slope of the curb in the vehicle coordinate system.

[0086] Exemplarily, according to the extrinsic parameters of the lidar, the coordinates of each point cloud in the lidar coordinate system are converted into coordinates in the vehicle coordinate system, taking the extrinsic parameters of the lidar as (x L , y L , z L , θ xL , θ yL , θ zL ) for example. L The extrinsic parameters matrix C L may be as shown in the following formula (9):

[0087] Wherein, x L , y L , z xL , θ yL , θ zL represent the three-dimensional coordinates, yaw angle, pitch angle and roll angle of the lidar in the vehicle coordinate system, respectively.

[0088] Taking the coordinates of the point cloud P L in the vehicle coordinate system as (XL, YL, ZL) for example, the coordinates of the point cloud P L in the lidar coordinate system can be converted into coordinates in the vehicle coordinate system according to the following formula (10):

[0089] It should be noted that the origin o of the laser radar coordinate system can be located at the center of the laser radar, and the x, y, and z axes are respectively the x, y, and z directions defined by the laser radar and following the x, y, and z directions of the base of the laser radar. In some implementations, a height constraint z can be set to filter out stray points that do not satisfy the height constraint z from the data points corresponding to the curb.

[0090] Exemplarily, after converting the point cloud corresponding to the curb into the vehicle coordinate system by the above formula (9) and formula (10), the data (data after the point cloud data is converted into the vehicle coordinate system) corresponding to the curb in the preset range of the vehicle (for example, the range of 5 to 30 meters in front of the vehicle) can be fitted, and one or more slopes of each curb in the vehicle coordinate system can be obtained.

[0091] In some implementations, the sampling grid shown in FIG. 7 is divided in the vehicle coordinate system, and a plurality of groups of data are selected based on the grid. The data corresponding to the curb in each group of data is clustered and fitted to obtain a slope value. More specifically, as shown in (a) and (b) of FIG. 7, the sampling grids are uniformly distributed on both sides of the X-axis of the vehicle coordinate system. The length of each sampling grid along the X-axis direction can be 5 meters, and the length along the Y-axis direction can be 9 meters, or 10 meters, or other values. The distance between the grid closest to the Y-axis and the Y-axis is s, which can be 6 meters, or 7 meters, or other values. Further, for each sampling grid, the corresponding data within a circle with the center of the grid as the center and a radius of r can be selected for clustering and fitting, where r can be 2 meters, or 2.5 meters. It can be understood that in actual implementation, the vehicle can not travel in a direction parallel to the lane line, or the road edge can not be straight, so that the data corresponding to the curb can not be selected in each grid. For example, when the curb is a curved curb, the position of the curb relative to the vehicle coordinate system can be as shown in (a) of FIG. 7. In this case, the curb corresponding data can only be selected in two grids, and in this case, the slope of the curb determined according to the frame point cloud data is not accurate, so the frame point cloud data can be discarded. When the curb corresponding data can be selected in multiple grids in the sampling grid, the data selected in each grid is clustered to obtain the curb corresponding data, and the clustered curb corresponding data is fitted to obtain the slope of the curb corresponding to the grid. Further, the slopes obtained from the samples selected according to the plurality of grids (such as the data selected from the five grids shown in (b) of FIG. 7) can be sorted, and the standard deviation of the plurality of slopes is calculated after removing the extreme values. When the standard deviation of the slope is less than or equal to 1.0, it can be considered that the curb of the current road section is straight or approximately straight, and the yaw angle of the lidar can be detected based on the curb of the road section. For example, when the standard deviation of the slope is less than or equal to 1.0, the average of the plurality of slopes can be taken as the slope of the curb of the road section in the vehicle coordinate system.

[0092] In actual implementation, the size of the foregoing sampling grid can be different in different scenarios, for example, the size of the sampling grid along the Y-axis direction can be adjusted according to the width of the road. Alternatively, in actual implementation, data selection in the vehicle coordinate system can also be performed in other ways.

[0093] S603, determining the angle by which the yaw angle of the lidar is offset according to the slope of the curb and the slope of the lane line determined based on the image.

[0094] Exemplarily, during the vehicle driving in a direction parallel to the lane line, the slope of the lane line is determined according to the mth frame of image, the slope of the curb with stable distribution reliability is determined according to the n th frame of point cloud data, and then the included angle a between the lane line and the X axis of the vehicle coordinate system is determined according to the slope of the lane line, the included angle a' between the curb and the X axis of the vehicle coordinate system is determined according to the slope of the curb, and the difference between the included angle a and the included angle a' is the angle offset by the yaw angle of the laser radar. Wherein, the time difference between the actual time when the camera device collects the mth frame of image and the starting time when the laser radar collects the n th frame of point cloud data is less than or equal to the time threshold, and exemplarily, the time threshold can be 0.05 seconds, or 0.04 seconds, or can also be other numerical values. In addition, whether the vehicle drives in a direction parallel to the lane line can be determined according to the method 300, which will not be described here.

[0095] In some implementations, the result determined according to the aforementioned mth frame of image and n th frame of point cloud data can be regarded as a single frame detection result. For example, as shown in (c) of FIG. 7, the dotted line indicates the position and slope of the lane line pixel in the vehicle coordinate system, the dashed line indicates the position and slope of the curb point cloud in the vehicle coordinate system, and the included angle a between the solid line and the dashed line is the angle offset by the yaw angle of the laser radar indicated by the single frame detection result. Further, the angle offset by the yaw angle of the laser radar can be determined according to the average value of the single frame detection results of the continuous 20 frames.

[0096] The sensor detection method provided by the embodiments of the present application can detect the yaw angle of the laser radar using the image collected by the camera device, has low computing resource consumption, good scene universality, can detect a larger range of yaw angles of the laser radar due to being not limited by the initial value, and has fast timeliness.

[0097] It can be understood that the aforementioned method 300 and method 600 are described by taking the camera device collecting the image in front of the vehicle and the laser radar collecting the point cloud data in front of the vehicle as examples, and in actual implementation, the yaw angle of the backward laser radar can also be verified by using the sensor detection method provided by the embodiments of the present application. When verifying the backward laser radar, the image in front of the vehicle can be used, or the image behind the vehicle can also be used for verification. In addition, the aforementioned method 300 and method 600 are described by taking the slope of the lane line and the curb in the vehicle coordinate system as examples, and in actual implementation, the pixels corresponding to the lane line and the point cloud data corresponding to the curb can also be converted to the world coordinate system, and then the slope of the lane line and the curb in the world coordinate system is determined.

[0098] In some implementations, when the yaw angle offset of the yaw angle of the laser radar is greater than or equal to a preset threshold, the vehicle can perform any one of the following: limit intelligent driving functions, or prompt the user to maintain the laser radar. Illustratively, the aforementioned intelligent driving functions can include, but are not limited to, the following functions that affect driving safety: auto parking assist (APA), auto valet parking (AVP), adaptive cruise control (ACC), lane cruise control (LCC), navigate cruise assist (NCA), front vehicle collision warning, lane departure warning, lane keeping assist, and rear vehicle collision warning. The NCA function refers to a function of controlling the vehicle to travel to a destination according to a navigation route, and being able to control the vehicle to pass through an intersection, change lanes, and change gears according to road information such as traffic lights.

[0099] For the implementation of prompting the user to maintain the laser radar, FIG. 8 shows a schematic diagram of a set of graphical user interfaces (GUIs) displayed by the vehicle display device. As shown in FIG. 8, the instrument screen of the vehicle displays a display area 820, a display area 830, and a display area 840. The display area 820 can display the current speed of the vehicle (e.g., 65 kilometers per hour (kph)), gear information (e.g., the current vehicle is in D gear), the minimum speed limit of the road (e.g., 40 kph), the maximum speed limit of the road (e.g., 80 kph), the remaining battery capacity (e.g., 80%), and the cruising range (e.g., 460 kilometers), etc. The display area 830 can display a virtual scene generated by the data collected by the sensor, which includes a vehicle icon and road environment information (e.g., information of other vehicles, information of lane lines, etc.) around the vehicle. Through the virtual scene, the user can confirm the relative position relationship between the vehicle and other vehicles. The display area 840 can display entertainment-related information, such as a music playing interface.

[0100] In some implementations, as shown in (a) of FIG. 8, when it is determined that the yaw angle of the lidar deviates by an angle greater than or equal to a preset threshold, the vehicle can control the instrument panel screen to display a pop-up window 831 including the text "Lidar abnormality, please repair". In yet other implementations, after it is determined that the yaw angle of the lidar deviates by an angle greater than or equal to a preset threshold, a request to start the LCC function is detected, the request is rejected, and the instrument panel screen is controlled to display a pop-up window 832 including the text "Lidar abnormality, unable to enable the lane keeping function". In still other implementations, a sound device such as a loudspeaker can also be used to play a related prompt, such as playing the voice "Lidar abnormality, please repair" through the loudspeaker at 850. Exemplarily, a light device can also be used to provide a prompt, for example, when it is determined that the yaw angle of the lidar deviates by an angle greater than or equal to a preset threshold, the ambient light in the vehicle can be controlled to display a red light or other warning light.

[0101] In actual implementation, when the yaw angle of the lidar deviates by an angle greater than or equal to a preset threshold, the virtual scene generated based on the sensors including the lidar can be inaccurate, and thus the virtual scene can not be displayed in the display area 830.

[0102] It should be noted that the elements in the interface shown in FIG. 8 above are only exemplary, and in actual implementation, when the user is prompted by the vehicle display device that the lidar is abnormal, the interface displayed by the vehicle display device can also include elements different from those shown in FIG. 8.

[0103] It should be further noted that the above describes the detection of whether the yaw angle of the lidar deviates and the prompt and control methods of the vehicle when the yaw angle of the lidar deviates. In actual implementation, the method for detecting the yaw angle of the 4D millimeter wave radar can refer to the descriptions in the foregoing methods 300 and 600, and specifically, the method for determining the curb slope based on the point cloud data of the 4D millimeter wave radar and the method for determining the yaw angle deviation of the 4D millimeter wave radar based on the curb slope and the lane line slope can refer to the descriptions in the method 600, and thus will not be described herein again.

[0104] FIG. 9 shows a schematic flowchart of a sensor detection method provided by an embodiment of the present application. The method 900 can be applied to the vehicle shown in FIG. 1, or the method can be executed by the system shown in FIG. 2. More specifically, the method includes:

[0105] S910, acquiring at least one frame of image collected by a first camera device in a first time period and a point cloud data set collected by a first radar in the first time period.

[0106] Exemplarily, the first camera can include the camera in the method 300, the first radar can be the laser radar or the 4D millimeter wave radar in the method 600, and the method of acquiring at least one frame of image and the point cloud data set is described with reference to the method 300 and the method 600, which will not be repeated here.

[0107] S920, determining, according to the at least one frame of image, a first slope of a lane line of the first road in which the vehicle is located in a first coordinate system.

[0108] The first coordinate system can be a vehicle coordinate system or a world coordinate system.

[0109] S930, determining, according to the point cloud data set, a second slope of a curb of the first road in the first coordinate system.

[0110] In some implementations, the at least one frame of image includes an Mth frame of image, the point cloud data set includes an Nth frame of point cloud data, and a time difference between a starting time of collecting the Mth frame of image and a starting time of collecting the Nth frame of point cloud data is less than or equal to a first threshold value; the first slope of the lane line of the first road in which the vehicle is located in the first coordinate system is determined according to the at least one frame of image, including: determining the first slope according to the Mth frame of image; the second slope of the curb of the first road in the first coordinate system is determined according to the point cloud data set, including: determining the second slope according to the Nth frame of point cloud data.

[0111] Exemplarily, the first threshold value and the aforementioned time length threshold value can be the same value, for example, the first threshold value can be 0.05 seconds, or can also be other time lengths. In addition, the specific implementation of determining the first slope according to one frame of image can refer to the description of the method 300, and the specific implementation of determining the second slope according to one frame of point cloud data can refer to the description of the method 600, which will not be repeated here.

[0112] In actual implementation, a plurality of angle values can be determined according to a plurality of sets of image and point cloud data, and a difference between a data collection time of each set of data in the plurality of sets of image and point cloud data is less than or equal to a first threshold value, and then a first angle is determined according to an average value of the plurality of angle values. More specific implementation can refer to the description of the method 600, which will not be repeated here.

[0113] In some implementations, the first group of data is determined according to the Nth frame of point cloud data, the first group of data indicating a position of point cloud data corresponding to the curb in the first coordinate system in the Nth frame of point cloud data; the first group of data is clustered and fitted to obtain a plurality of curb data; a slope fitting is performed on each piece of curb data in the plurality of curb data to obtain a plurality of slope values, each slope value in the plurality of slope values being a corresponding slope in the plurality of curb data; and the second slope is determined according to the plurality of slope values.

[0114] Exemplarily, each piece of curb data can be selected in a sampling grid, and the method for determining the slope value and the method for determining the second slope value can refer to the description of the method 600, which will not be described here again.

[0115] S940, determining, according to the first slope and the second slope, a first angle by which the yaw angle of the first radar deviates.

[0116] In some implementations, S940 can be refined as: when the standard deviation corresponding to the plurality of slope values is less than or equal to a preset threshold, determining, according to the first slope and the second slope, the first angle.

[0117] Exemplarily, the preset threshold can be 1.0, or can also be other numerical values. If the standard deviation corresponding to the plurality of slope values is greater than the preset threshold, the second slope is discarded or ignored, that is, the second slope is no longer used to determine the angle by which the yaw angle of the first radar deviates.

[0118] In some implementations, at least one frame of image and point cloud data are collected during the vehicle driving in the first road, and S940 can be refined as: when the first slope indicates that the included angle between the lane line and the driving direction of the vehicle is less than or equal to a first included angle threshold, determining, according to the first slope and the second slope, the first angle.

[0119] Exemplarily, the first included angle threshold can be 5°, or can also be 3°, or can also be other numerical values.

[0120] In some implementations, the method further includes: when the first angle is greater than or equal to a second included angle threshold, controlling a prompt device to prompt first information, the first information being used to prompt that the pose of the first radar is abnormal and / or to maintain the first radar.

[0121] Exemplarily, the second included angle threshold can be 10°, or can also be 5°, or can also be other numerical values. The prompt device can include one or more of the vehicle-mounted display device, the vehicle-mounted sound device, and the vehicle-mounted light device in the foregoing embodiments, and the first information can include the information in the pop-up window 831 shown in FIG. 8, or can also include voice information played by the loudspeaker at 850.

[0122] In some implementations, the method further includes: when the first angle is greater than or equal to a second included angle threshold, limiting the enabling of the intelligent driving function.

[0123] Exemplarily, the intelligent driving function can include one or more of the intelligent driving functions in the foregoing embodiments.

[0124] The sensor detection method provided by the embodiments of the present application can quickly and accurately detect whether the yaw angle of the first radar in the vehicle deviates from the calibration value, and in the case that the yaw angle of the first radar deviates too much, relevant information can be prompted and / or the intelligent driving function can be limited, which helps to improve the safety of the vehicle.

[0125] In each of the embodiments of the present application, the terms and / or descriptions of different embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0126] The method provided by the embodiments of the present application is described in detail above in combination with FIG. 1 to FIG. 9. The device provided by the embodiments of the present application will be described in detail below in combination with FIG. 10 and FIG. 11. It should be understood that the description of the device embodiments corresponds to the description of the method embodiments, and therefore, the content not described in detail can be referred to the method embodiments described above, and for brevity, will not be described here again.

[0127] FIG. 10 shows a schematic block diagram of the sensor detection device 2000 provided by the embodiments of the present application, which can include units for performing the methods described in the foregoing embodiments. And each unit in the device 2000 is used to implement the corresponding flow of the method embodiments described above. The device 2000 includes an acquisition unit 2010, which can be used to implement the corresponding data acquisition or transceiving function. The device 2000 further includes a processing unit 2020, which can be used to implement the corresponding processing function.

[0128] Optionally, the device 2000 further includes a storage unit, which can be used to store instructions and / or data, and the processing unit 2020 can read the instructions and / or data in the storage unit to enable the device to implement the related actions in the foregoing method embodiments.

[0129] It should be understood that the specific process of each unit executing the corresponding steps described above has been described in detail in the method embodiments described above, and for brevity, will not be described here again.

[0130] It should also be understood that the device 2000 here is embodied in the form of functional units. The term "module" or "unit" here can refer to an application-specific ASIC, an electronic circuit, a processor (for example, a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combination logic circuit, and / or other suitable components that support the described functions.

[0131] The apparatus in this embodiment has the function of implementing the corresponding steps in the foregoing method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the acquisition unit 2010 can be replaced by a transceiver, and other units, such as the processing unit, can be replaced by a processor, for performing the related processing operations in each method embodiment.

[0132] For example, the acquisition unit 2010 and the processing unit 2020 can be arranged in the vehicle 100 shown in FIG. 1, or can also be arranged in the system shown in FIG. 2. More specifically, the acquisition unit 2010 and the processing unit 2020 can be arranged in the detection module 220. For example, the operations performed by the acquisition unit 2010 and the processing unit 2020 can be performed by one processor, or can also be performed by different processors. In a specific implementation process, the one or more processors can be the processor arranged in the vehicle 100 shown in FIG. 1; or the apparatus 2000 can be a chip arranged in the vehicle 100.

[0133] In a specific implementation process, the units in the above apparatus can be integrated together or can also be independently implemented. In one implementation, the units are integrated together to be implemented in the form of a system on a chip (SoC).

[0134] FIG. 11 is another schematic block diagram of a sensor detection apparatus provided by an embodiment of the present application. The apparatus 2100 shown in FIG. 11 can include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, the transceiver 2120, and the memory 2130 are connected through an internal connection path. The memory 2130 is configured to store instructions, and the processor 2110 is configured to execute the instructions stored in the memory 2130 to implement the methods in the above embodiments. Optionally, the memory 2130 can be coupled to the processor 2110 through an interface, or can be integrated with the processor 2110.

[0135] It should be noted that the transceiver 2120 can include, but is not limited to, a transceiving device such as an input / output interface, to implement the communication between the apparatus 2100 and other devices or communication networks.

[0136] The memory 2130 can be volatile memory and / or nonvolatile memory. The nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory, for example. The volatile memory can be random access memory (RAM), for example. The RAM can be external cache memory, for example. As examples without limitation, the RAM includes the following types: static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The transceiver 2120 uses a transceiving device such as, but not limited to, a transceiver to implement communication between the device 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above-described embodiments.

[0138] The embodiments of the present application also provide a vehicle, which includes the device 2000 or the device 2100 in the above-described embodiments.

[0139] The embodiments of the present application also provide a computer program product, which includes computer program codes, and when the computer program codes are run on a computer, the computer is caused to implement the methods in the above-described embodiments of the present application.

[0140] The embodiments of the present application also provide a computer readable storage medium, which stores computer instructions, and when the computer instructions are run on a computer, the computer is caused to implement the methods in the above-described embodiments of the present application.

[0141] The embodiments of the present application also provide a chip, which includes a circuit for implementing the methods in the above-described embodiments of the present application.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0143] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein is a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0144] In the embodiments of the present application, the prefix words such as "first", "second" are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of ordinal words such as ordinal words in the embodiments of the present application does not constitute a limitation on the described objects, and the description of the described objects should be referred to the description of the context in the claims or embodiments, and should not constitute redundant limitations because of the use of such prefix words.

[0145] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0146] In various embodiments of the present application, the terms and / or descriptions of various embodiments have consistency and can be mutually referred to, unless otherwise specified and logically conflicted, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0147] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0148] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A sensor detection method, characterized by, The method comprises: acquiring at least one image frame collected by a first camera in a first time period and point cloud data collected by a first radar in the first time period, the vehicle comprising the first camera and the first radar; determining a first slope of a lane line of a first road where the vehicle is located in a first coordinate system according to the at least one image frame, the first coordinate system being a vehicle coordinate system or a world coordinate system; determining a second slope of a curb of the first road in the first coordinate system according to the point cloud data; determining a first angle by which a yaw angle of the first radar is offset according to the first slope and the second slope.

2. The method of claim 1, wherein, The at least one image frame comprises an Mth image frame, and the point cloud data comprises Nth point cloud data, and a time difference between a starting time of collecting the Mth image frame and a starting time of collecting the Nth point cloud data is less than or equal to a first threshold value; The method further comprises: determining the first slope according to the Mth image frame; The method further comprises: determining the second slope according to the Nth point cloud data.

3. The method of claim 2, wherein, The method further comprises: determining a first set of data indicating positions of point cloud data corresponding to the curb in the first coordinate system in the Nth point cloud data according to the Nth point cloud data; performing clustering and fitting on the first set of data to obtain a plurality of curb data; performing slope fitting on each piece of curb data in the plurality of curb data to obtain a plurality of slope values, each slope value in the plurality of slope values being a corresponding slope in the plurality of curb data; determining the second slope according to the plurality of slope values.

4. The method of claim 3, wherein, The method further comprises: when a standard deviation corresponding to the plurality of slope values is less than or equal to a preset threshold value, determining the first angle according to the first slope and the second slope.

5. The method according to any one of claims 1 to 4, characterized in that, The at least one image frame and the point cloud data are collected during driving of the vehicle on the first road, and the method further comprises: when an included angle between the lane line and a driving direction of the vehicle indicated by the first slope is less than or equal to a first included angle threshold value, determining the first angle according to the first slope and the second slope.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: when the first angle is greater than or equal to a second included angle threshold value, controlling a prompt device to prompt first information, the first information being used to prompt that a pose of the first radar is abnormal and / or to maintain the first radar.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: when the first angle is greater than or equal to a second included angle threshold value, limiting enabling of an intelligent driving function.

8. A sensor detection device, characterized by The method comprises: The acquisition unit is configured to acquire at least one image frame collected by a first camera in a first time period and point cloud data collected by a first radar in the first time period, and the vehicle comprises the first camera and the first radar. The processing unit is configured to: determine a first slope of a lane line of a first road in which the vehicle is located in a first coordinate system according to the at least one image frame, the first coordinate system comprising a vehicle coordinate system or a world coordinate system; determine a second slope of a curb of the first road in the first coordinate system according to the point cloud data; determine a first angle by which a yaw angle of the first radar deviates according to the first slope and the second slope.

9. The apparatus of claim 8, wherein, The at least one image frame comprises an Mth image frame, and the point cloud data comprises Nth point cloud data, and a time difference between a starting time of collecting the Mth image frame and a starting time of collecting the Nth point cloud data is less than or equal to a first threshold value. The processing unit is configured to: determine the first slope according to the Mth image frame; determine the second slope according to the Nth point cloud data.

10. The apparatus of claim 9, wherein, The processing unit is configured to: determine a first group of data according to the Nth point cloud data, the first group of data indicating positions of point cloud data corresponding to the curb in the Nth point cloud data in the first coordinate system; cluster and fit the first group of data to obtain a plurality of curb data; perform slope fitting on each piece of curb data in the plurality of curb data to obtain a plurality of slope values, each slope value in the plurality of slope values being a corresponding slope in the plurality of curb data; determine the second slope according to the plurality of slope values.

11. The apparatus of claim 10, wherein, The processing unit is configured to: when a standard deviation corresponding to the plurality of slope values is less than or equal to a preset threshold value, determine the first angle according to the first slope and the second slope.

12. The apparatus of any one of claims 8-11, wherein, The at least one image frame and the point cloud data are collected during driving of the vehicle on the first road, and the processing unit is configured to: when the first slope indicates that an included angle between the lane line and a driving direction of the vehicle is less than or equal to a first included angle threshold value, determine the first angle according to the first slope and the second slope.

13. The apparatus of any one of claims 8-12, wherein, The processing unit is further configured to: when the first angle is greater than or equal to a second included angle threshold value, control a prompt device to prompt first information, the first information being used to prompt that a pose of the first radar is abnormal and / or to maintain the first radar.

14. The apparatus of any one of claims 8-13, wherein, The processing unit is further configured to: when the first angle is greater than or equal to a second included angle threshold value, limit enabling of an intelligent driving function.

15. A sensor detection device, characterized by The apparatus comprises: a processor configured to execute a computer program stored in a memory to cause the apparatus to perform the method of any one of claims 1 to 7.

16. The apparatus of claim 15, wherein, The apparatus further comprises the memory.

17. A computer-readable storage medium, characterized in that, instructions stored thereon, which, when executed by a processor, implement the method of any one of claims 1 to 7.

18. A chip, characterized by The chip comprises a circuit configured to perform the method of any one of claims 1 to 7.

19. A computer program product, characterised in that, The computer program product comprises computer program code which, when run by a processor, implements the method according to any one of claims 1 to 7.

20. A vehicle characterized by The vehicle comprises the apparatus according to any one of claims 8 to 16, or the computer-readable storage medium according to claim 17, or the chip according to claim 18, or the vehicle is loaded with the computer program product according to claim 19.

Images