Fault diagnosis and treatment method, device, medium and vehicle for vehicle-mounted laser radar

Abstract

The present invention relates to the field of autonomous driving technology, and specifically provides a method, device, medium and vehicle for diagnosing and processing faults of a vehicle-mounted laser radar, aiming to solve the problem of how to diagnose faults existing in the vehicle-mounted laser radar in a timely and accurate manner, and process them in real time. To this end, the present invention can compare the real-time point cloud data collected by the vehicle-mounted laser radar for a first preset target with the pre-stored standard point cloud data of the first preset target, determine whether the vehicle-mounted laser radar has a first type of fault based on the comparison result, and determine whether to process the first type of fault based on the current scene in which the vehicle is located. By combining the current scene in which the vehicle is located to determine whether to process the first type of fault, the vehicle-mounted laser radar can be diagnosed in a timely and accurate manner, and effectively processed, which can ensure the effective diagnosis and troubleshooting of the vehicle-mounted laser radar, further ensure the function of autonomous driving, and improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and specifically provides a method, device, medium and vehicle for diagnosing and processing faults of a vehicle-mounted laser radar. Background Art

[0002] With the continuous development of autonomous driving technology, the related functions and user experience of autonomous driving have been greatly improved. As a core component of autonomous driving technology, the ability of on-board LiDAR to collect information about the surrounding environment in real time and accurately is crucial to the autonomous driving function. Therefore, it is necessary to effectively diagnose and troubleshoot on-board LiDAR.

[0003] In the prior art, instruments are mainly used to diagnose whether the vehicle-mounted laser radar has a fault and to repair the fault. This method cannot immediately feedback the problem of the vehicle-mounted laser radar and calibrate the parameters of the vehicle-mounted laser radar in real time.

[0004] Accordingly, the art needs a new vehicle-mounted lidar fault diagnosis and processing solution to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present invention is proposed to provide a solution or at least partially solve the problem of how to promptly and accurately diagnose the faults existing in the vehicle-mounted laser radar and handle them in real time.

[0006] In a first aspect, the present invention provides a fault diagnosis and processing method for a vehicle-mounted laser radar, comprising:

[0007] Compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target;

[0008] Determine whether the vehicle-mounted laser radar has a first type of fault according to the comparison result;

[0009] Get the current scene where the vehicle is located;

[0010] When the first type of fault exists, it is determined whether to handle the first type of fault according to the current scenario of the vehicle.

[0011] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, the real-time point cloud data collected by the vehicle-mounted laser radar for a first preset target is compared with the pre-stored standard point cloud data of the first preset target, including:

[0012] Comparing the real-time point cloud data with the standard point cloud data to obtain the similarity between the real-time point cloud data and the standard point cloud data;

[0013] The determining, according to the comparison result, whether the vehicle-mounted laser radar has a first type of fault includes:

[0014] When the similarity is less than a preset similarity threshold, it is determined that the vehicle-mounted laser radar has the first type of fault.

[0015] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, when the first type of fault exists, judging whether to process the first type of fault according to the current scene of the vehicle includes:

[0016] Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors;

[0017] If it is determined that the current scene of the vehicle is not a preset scene, a fault alarm message is sent to the vehicle-mounted device;

[0018] If it is determined that the current scene in which the vehicle is located is a preset scene, no further processing is performed.

[0019] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, when the first type of fault exists, judging whether to process the first type of fault according to the current scene of the vehicle includes:

[0020] Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors;

[0021] If it is determined that the current scene of the vehicle is not a preset scene, determining the fault duration of the first type of fault;

[0022] If the fault duration exceeds a preset duration threshold, a fault alarm message is sent to the vehicle-mounted device;

[0023] When the fault duration does not exceed the preset duration threshold, no further processing is performed;

[0024] If it is determined that the current scene in which the vehicle is located is a preset scene, no further processing is performed.

[0025] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, the obtaining of the current scene of the vehicle includes:

[0026] The real-time vehicle-side signal and the real-time image data collected by the vehicle-mounted camera are integrated in real time;

[0027] The current scene of the vehicle is obtained according to the result of the real-time fusion.

[0028] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, the real-time vehicle-side signal includes at least one of a vehicle-side CAN signal, an RTK signal and an IMU signal.

[0029] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, it also includes:

[0030] Acquire real-time point cloud data and real-time image data collected by the vehicle-mounted laser radar and the vehicle-mounted camera for a second preset target respectively;

[0031] By comparing the real-time point cloud data and the real-time image data, determining whether the vehicle-mounted laser radar has a second type of fault;

[0032] If it is determined that the vehicle-mounted laser radar has the second type of fault, the second type of fault is repaired.

[0033] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, the comparison of the real-time point cloud data and the real-time image data to determine whether the vehicle-mounted laser radar has a second type of fault includes:

[0034] Acquire first position information of the second preset target according to the real-time point cloud data;

[0035] Acquire second position information of the second preset target according to the real-time image data;

[0036] Comparing the first position information with the second position information according to external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera to obtain a position deviation between the first position information and the second position information;

[0037] If the position deviation is greater than a preset deviation threshold, it is determined that the vehicle-mounted laser radar has the second type of fault.

[0038] In a technical solution of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method, if it is determined that the vehicle-mounted laser radar has the second type of fault, the second type of fault is repaired, including:

[0039] Step S1: performing real-time vehicle-side calibration on the external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera according to the real-time point cloud data and the real-time image data;

[0040] Step S2: Determine whether the real-time vehicle-side calibration of the external parameter is successful; if so, jump to step S3; if not, jump to step S4;

[0041] Step S3: completing the repair of the second type of fault;

[0042] Step S4: Determine whether the number of real-time vehicle-side calibrations is greater than a preset number threshold; if so, jump to step S5; if not, jump to step S1;

[0043] Step S5: Control the cloud server connected to the vehicle to send an updated calibration file to the vehicle to complete the repair of the second type of fault, wherein the updated calibration file includes the accurate internal parameters of the vehicle-mounted laser radar and the accurate external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera.

[0044] In a second aspect, a control device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the vehicle-mounted laser radar fault diagnosis and processing method described in any technical solution of one of the technical solutions of the above-mentioned vehicle-mounted laser radar fault diagnosis and processing method.

[0045] In a third aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the vehicle-mounted laser radar fault diagnosis and processing method described in any one of the technical solutions of the vehicle-mounted laser radar fault diagnosis and processing method.

[0046] In a fourth aspect, a vehicle is provided, comprising the control device described in the above-mentioned control device technical solution.

[0047] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0048] In the technical solution of the present invention, the present invention can compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target, and judge whether the vehicle-mounted laser radar has a first type of fault according to the comparison result, and judge whether to handle the first type of fault according to the current scene where the vehicle is located. Through the above configuration, the present invention can diagnose whether the vehicle-mounted laser radar has a first type of fault in real time, and judge whether to handle the first type of fault in combination with the current scene where the vehicle is located, and can realize timely and accurate diagnosis of the vehicle-mounted laser radar, and perform effective processing, which can ensure the effective diagnosis and troubleshooting of the vehicle-mounted laser radar, further ensure the function of automatic driving, and improve the user experience.

[0049] Solution 1. A method for diagnosing and processing a fault of a vehicle-mounted laser radar, characterized by comprising:

[0050] Compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target;

[0051] Determine whether the vehicle-mounted laser radar has a first type of fault according to the comparison result;

[0052] Get the current scene where the vehicle is located;

[0053] When the first type of fault exists, it is determined whether to handle the first type of fault according to the current scenario of the vehicle.

[0054] Solution 2. The method according to Solution 1, characterized in that:

[0055] The comparing the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target includes:

[0056] Comparing the real-time point cloud data with the standard point cloud data to obtain the similarity between the real-time point cloud data and the standard point cloud data;

[0057] The determining, according to the comparison result, whether the vehicle-mounted laser radar has a first type of fault includes:

[0058] When the similarity is less than a preset similarity threshold, it is determined that the vehicle-mounted laser radar has the first type of fault.

[0059] Solution 3. The method according to solution 1 or 2 is characterized in that when the first type of fault exists, judging whether to handle the first type of fault according to the current scene of the vehicle includes:

[0060] Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors;

[0061] If it is determined that the current scene of the vehicle is not a preset scene, a fault alarm message is sent to the vehicle-mounted device;

[0062] If it is determined that the current scene in which the vehicle is located is a preset scene, no further processing is performed.

[0063] Solution 4. The method according to solution 1 or 2 is characterized in that when the first type of fault exists, judging whether to handle the first type of fault according to the current scene of the vehicle includes:

[0064] Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors;

[0065] If it is determined that the current scene of the vehicle is not a preset scene, determining the fault duration of the first type of fault;

[0066] If the fault duration exceeds a preset duration threshold, a fault alarm message is sent to the vehicle-mounted device;

[0067] When the fault duration does not exceed the preset duration threshold, no further processing is performed;

[0068] If it is determined that the current scene in which the vehicle is located is a preset scene, no further processing is performed.

[0069] Solution 5. The method according to Solution 1 is characterized in that obtaining the current scene of the vehicle comprises:

[0070] The real-time vehicle-side signal and the real-time image data collected by the vehicle-mounted camera are integrated in real time;

[0071] The current scene of the vehicle is obtained according to the result of the real-time fusion.

[0072] Solution 6. The method according to Solution 5 is characterized in that the real-time vehicle-side signal includes at least one of a vehicle-side CAN signal, an RTK signal and an IMU signal.

[0073] Solution 7. The method according to Solution 1, further comprising:

[0074] Acquire real-time point cloud data and real-time image data collected by the vehicle-mounted laser radar and the vehicle-mounted camera for a second preset target respectively;

[0075] By comparing the real-time point cloud data and the real-time image data, determining whether the vehicle-mounted laser radar has a second type of fault;

[0076] If it is determined that the vehicle-mounted laser radar has the second type of fault, the second type of fault is repaired.

[0077] Solution 8. The method according to Solution 7 is characterized in that the step of comparing the real-time point cloud data with the real-time image data to determine whether the vehicle-mounted laser radar has a second type of fault comprises:

[0078] Acquire first position information of the second preset target according to the real-time point cloud data;

[0079] Acquire second position information of the second preset target according to the real-time image data;

[0080] Comparing the first position information with the second position information according to external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera to obtain a position deviation between the first position information and the second position information;

[0081] If the position deviation is greater than a preset deviation threshold, it is determined that the vehicle-mounted laser radar has the second type of fault.

[0082] Solution 9. The method according to Solution 7 is characterized in that if it is determined that the vehicle-mounted laser radar has the second type of fault, repairing the second type of fault includes:

[0083] Step S1: performing real-time vehicle-side calibration on the external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera according to the real-time point cloud data and the real-time image data;

[0084] Step S2: Determine whether the real-time vehicle-side calibration of the external parameter is successful; if so, jump to step S3; if not, jump to step S4;

[0085] Step S3: completing the repair of the second type of fault;

[0086] Step S4: Determine whether the number of real-time vehicle-side calibrations is greater than a preset number threshold; if so, jump to step S5; if not, jump to step S1;

[0087] Step S5: Control the cloud server connected to the vehicle to send an updated calibration file to the vehicle to complete the repair of the second type of fault, wherein the updated calibration file includes the accurate internal parameters of the vehicle-mounted laser radar and the accurate external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera.

[0088] Solution 10. A control device, comprising a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and is characterized in that the program codes are suitable for being loaded and run by the processor to execute the fault diagnosis and processing method of the vehicle-mounted laser radar described in any one of Solutions 1 to 9.

[0089] Solution 11. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are suitable for being loaded and run by a processor to execute the fault diagnosis and processing method of the vehicle-mounted laser radar described in any one of Solutions 1 to 9.

[0090] Solution 12. A vehicle, characterized in that the vehicle includes the control device described in Solution 10. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0092] Figure 1 1. It is a flowchart of the main steps of a method for diagnosing and processing a fault of a vehicle-mounted laser radar according to an embodiment of the present invention;

[0093] Figure 2 1 is a flow chart of the main steps of a method for diagnosing and handling a second type of fault of a vehicle-mounted laser radar according to an embodiment of the present invention.

[0094] Figure 3 1 is a flow chart of main steps of a method for diagnosing and processing a fault of a vehicle-mounted laser radar according to an implementation of an embodiment of the present invention;

[0095] Figure 4 It is a main system structure block diagram of a system for implementing a fault diagnosis and processing method for a vehicle-mounted laser radar according to an implementation of an embodiment of the present invention. DETAILED DESCRIPTION

[0096] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0097] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0098] See attached Figure 1 , Figure 1FIG. 1 is a flow chart showing the main steps of a method for diagnosing and processing a fault of a vehicle-mounted laser radar according to an embodiment of the present invention. Figure 1 As shown, the fault diagnosis and processing method of the vehicle-mounted laser radar in the embodiment of the present invention mainly includes the following steps S101-S104.

[0099] Step S101: Compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target.

[0100] In this embodiment, a first preset target can be set, and the real-time point cloud data of the first preset target collected by the vehicle-mounted laser radar is compared with the standard point cloud data of the first preset target pre-stored on the vehicle side.

[0101] In one implementation, the first preset target may be a relatively common target in a vehicle driving scene, such as a relatively common vehicle model (such as a three-compartment five-seater sedan).

[0102] Step S102: Determine whether the vehicle-mounted laser radar has a first type of fault based on the comparison result.

[0103] In this embodiment, it can be determined whether the vehicle-mounted laser radar has a first type of fault based on the comparison result between the real-time point cloud data and the standard point cloud data.

[0104] In one embodiment, the first type of fault may be that the on-board laser radar is obstructed or the real-time point cloud data collected by the on-board laser radar contains noise, resulting in inaccurate or incomplete real-time point cloud data collected by the on-board laser radar.

[0105] Step S103: Acquire the current scene of the vehicle.

[0106] In this embodiment, the current scene in which the vehicle is located can be further acquired.

[0107] In one embodiment, the current scene in which the vehicle is located can be a scene differentiated by weather, such as a rainy and snowy weather scene, a foggy scene, and a sunny scene; it can also be a scene differentiated by time, such as a day scene and a night scene; it can also be a scene differentiated by location, such as a tunnel scene, a basement scene, an ordinary road scene, etc.

[0108] Step S104: When a first type fault exists, determine whether to handle the first type fault according to the current scenario of the vehicle.

[0109] In this embodiment, when it is determined that the vehicle-mounted laser radar has a first type of fault, it can be determined whether to handle the first type of fault according to the current scene the vehicle is in. That is, a comprehensive judgment is made on the first type of fault in combination with the scene to determine whether to handle the first type of fault.

[0110] Based on the above steps S101 to S104, the embodiment of the present invention can compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target, and determine whether the vehicle-mounted laser radar has a first type of fault based on the comparison result, and determine whether to handle the first type of fault based on the current scene where the vehicle is located. Through the above configuration, the embodiment of the present invention can diagnose whether the vehicle-mounted laser radar has a first type of fault in real time, and determine whether to handle the first type of fault based on the current scene where the vehicle is located, and can achieve timely and accurate diagnosis of the vehicle-mounted laser radar and effective processing, which can ensure the effective diagnosis and troubleshooting of the vehicle-mounted laser radar, further ensure the function of autonomous driving, and improve user experience.

[0111] Steps S101 to S104 are further described below.

[0112] In one implementation of the embodiment of the present invention, step S101 may further include the following steps:

[0113] Compare the real-time point cloud data with the standard point cloud data to obtain the similarity between the real-time point cloud data and the standard point cloud data.

[0114] In this embodiment, the real-time point cloud data of the first preset target may be compared with the standard point cloud data to obtain the similarity between the real-time point cloud data and the standard point cloud data.

[0115] In one implementation, an ICP (Iterative Closest Point) algorithm may be used to obtain the similarity between real-time point cloud data and standard point cloud data. An NDT (Normal Distributions Transform) algorithm may also be used to obtain the similarity between real-time point cloud data and standard point cloud data.

[0116] In one implementation of the embodiment of the present invention, step S102 may further include the following steps:

[0117] When the similarity is less than a preset similarity threshold, it is determined that the vehicle-mounted laser radar has a first type of fault.

[0118] In this embodiment, the similarity between the acquired real-time point cloud data and the standard point cloud data can be compared with a preset similarity threshold. When the similarity is less than the similarity threshold, it is determined that the vehicle-mounted laser radar has a first type of fault.

[0119] Specifically, since the real-time point cloud data and the standard point cloud data collected by the vehicle-mounted laser radar are both point cloud data of the same type of target, when the vehicle-mounted laser radar works normally and the environmental factors of the current scene where the vehicle is located do not cause the point cloud data to be missing, the real-time point cloud data and the standard point cloud data should have a high similarity; when the similarity between the real-time point cloud data and the standard point cloud data is less than the similarity threshold, it can be considered that the vehicle-mounted laser radar has a first type of failure or the environmental factors of the current scene where the vehicle is located will cause the point cloud data to be missing. Those skilled in the art can set the value of the similarity threshold according to the needs of actual applications.

[0120] In one implementation of the embodiment of the present invention, step S103 may further include the following steps S1031 and S1032:

[0121] Step S1031: Fuse the real-time vehicle-side signal of the vehicle and the real-time image data collected by the vehicle-mounted camera in real time.

[0122] Step S1032: Obtain the current scene of the vehicle based on the result of real-time fusion.

[0123] In this embodiment, real-time fusion can be performed based on the real-time vehicle-side signal of the vehicle and the real-time image data collected by the vehicle-mounted camera, and the current scene of the vehicle can be obtained based on the result of the real-time fusion. That is, when obtaining the vehicle scene, only the real-time vehicle-side signal and the real-time image data collected by the vehicle-mounted camera are used to obtain the current scene of the vehicle, and the real-time point cloud data collected by the vehicle-mounted laser radar is not used when obtaining the current scene, so as to avoid errors in the judgment of the current scene of the vehicle when the vehicle-mounted laser radar fails.

[0124] In one embodiment, the real-time vehicle-side signal may include at least one of a vehicle-side CAN (Controller Area Network) signal, an RTK (Real-time kinematic) signal, and an IMU (Inertial Measurement Unit) signal.

[0125] In one implementation, a convolutional neural network-based model can be used for feature extraction, and then a region candidate network and a detection head can be used for target detection to complete the perception algorithm. Finally, the real-time vehicle-side signal and the real-time image data collected by the on-board camera can be fused in real time.

[0126] In one implementation of the embodiment of the present invention, step S104 may further include the following steps S1041 and S1043:

[0127] Step S1041: Determine whether the current scene in which the vehicle is located is a preset scene, where the preset scene is a scene in which point cloud data is missing due to environmental factors.

[0128] Step S1042: If it is determined that the current scene of the vehicle is not a preset scene, a fault alarm message is sent to the vehicle-mounted device;

[0129] Step S1043: If it is determined that the current scene of the vehicle is a preset scene, no further processing is performed.

[0130] In this embodiment, it can be determined whether the current scene in which the vehicle is located is a preset scene, that is, a scene in which point cloud data is missing due to environmental factors, such as rainy and snowy scenes, foggy scenes, night scenes, tunnel scenes, basement scenes, etc. Since there will be obstacles or insufficient light in the preset scene itself, the real-time point cloud data collected by the laser radar will be missing or there will be noise. When the current scene in which the vehicle is located is a preset scene, it can be considered that the first type of fault in the vehicle-mounted laser radar is caused by the preset scene, and there is no need to further process the first type of fault. When the current scene in which the vehicle is located is not a preset scene, it is considered that the first type of fault in the vehicle-mounted laser radar is caused by the vehicle-mounted laser radar itself. Then, it is necessary to send a fault alarm message to the vehicle-mounted device, such as sending a voice fault alarm message, displaying the fault alarm message on the vehicle's dashboard, or uploading the fault alarm message to a cloud server connected to the vehicle, etc., to prompt the driver or the corresponding after-sales personnel to perform corresponding processing, such as cleaning foreign objects in front of the vehicle-mounted laser radar, or repairing the vehicle-mounted laser radar.

[0131] In another implementation of the embodiment of the present invention, step S104 may further include the following steps S1044 and S1048:

[0132] Step S1044: determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors;

[0133] Step S1045: If it is determined that the current scene of the vehicle is not a preset scene, determining the fault duration of the first type of fault;

[0134] Step S1046: If the fault duration exceeds the preset duration threshold, a fault alarm message is sent to the vehicle-mounted device;

[0135] Step S1047: When the fault duration does not exceed the preset duration threshold, no further processing is performed;

[0136] Step S1048: If it is determined that the current scene of the vehicle is a preset scene, no further processing is performed.

[0137] In this embodiment, after determining that a first type of fault exists in the vehicle-mounted laser radar, a comprehensive determination is made as to whether to handle the first type of fault based on the current scenario in which the vehicle is located and the duration of the first type of fault.

[0138] Specifically, when the current scene in which the vehicle is located is not a preset scene and the fault duration is greater than the preset duration threshold, a fault alarm message can be sent to the vehicle-mounted equipment. When the current scene in which the vehicle is located is not a preset scene or the fault duration of the first type of fault is less than the preset duration threshold, the first type of fault is not processed. In other words, when it is determined that the vehicle-mounted laser radar has a first type of fault, the current scene and the fault duration will be comprehensively considered. If the current scene is a rainy and snowy scene, a foggy scene, a night scene, a tunnel scene, a basement scene, etc., it can be considered that the first type of fault is caused by the preset scene; or when there are some temporary obstructions in front of the vehicle-mounted laser radar and cause a first type of fault, the first type of fault is not processed immediately, but waits for the preset duration threshold. If the obstruction still exists, the first type of fault is further processed.

[0139] In one implementation of the present invention, see the attached Figure 2 , Figure 2 It is a flowchart diagram of the main steps of a method for diagnosing and handling a second type of fault of a vehicle-mounted laser radar according to an implementation of an embodiment of the present invention.

[0140] like Figure 2 As shown, in addition to the above steps S101 to S104, the present invention may also include the following steps S105 to S107:

[0141] Step S105: Acquire real-time point cloud data and real-time image data collected by the vehicle-mounted laser radar and the vehicle-mounted camera for the second preset target respectively.

[0142] In this embodiment, the real-time point cloud data and real-time image data of the vehicle-mounted laser radar and the vehicle-mounted camera for the same second preset target can be obtained. The second preset target can be the same target in the same target acquisition area of ​​the vehicle-mounted laser radar and the vehicle-mounted camera. A target with a relatively obvious shape and outline in the same target acquisition area can be selected as the second preset target, such as a vehicle license plate, a signal light, etc.

[0143] Step S106: Determine whether the vehicle-mounted laser radar has a second type of fault by comparing the real-time point cloud data and the real-time image data.

[0144] In this embodiment, the second type of fault may be inaccurate internal and external parameters of the vehicle-mounted laser radar, where the internal parameter refers to the conversion relationship between the laser emitter coordinate system and the vehicle-mounted laser radar's own coordinate system; the external parameter refers to the conversion relationship between the vehicle-mounted laser radar's own coordinate system and the vehicle-mounted camera coordinate system.

[0145] In one implementation, step S106 may further include the following steps S1061 to S1064:

[0146] Step S1061: Acquire first position information of a second preset target according to real-time point cloud data;

[0147] Step S1062: Acquire second position information of a second preset target according to the real-time image data;

[0148] Step S1063: comparing the first position information and the second position information according to external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera, and obtaining a position deviation between the first position information and the second position information;

[0149] Step S1064: If the position deviation is greater than the preset deviation threshold, it is determined that the vehicle-mounted laser radar has a second type of fault.

[0150] In this embodiment, the position coordinates (first position information) of the second preset target in the vehicle-mounted laser radar's own coordinate system can be obtained based on the real-time point cloud data. The position coordinates (second position information) of the second preset target in the vehicle-mounted camera coordinate system can be obtained based on the real-time image data. The first position information can be converted to the vehicle-mounted camera coordinate system based on the external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera to obtain the position coordinates of the first position information in the vehicle-mounted camera coordinate system, and the position coordinates of the first position information in the vehicle-mounted camera coordinate system can be compared with the second position information to obtain the position deviation between the first position information and the second position information. If the position deviation is greater than the preset deviation threshold, it can be considered that the vehicle-mounted laser radar has a second type of fault, that is, the internal and external parameters of the vehicle-mounted laser radar are inaccurate.

[0151] Step S107: If it is determined that the vehicle-mounted laser radar has a second type of fault, the second type of fault is repaired.

[0152] In this embodiment, step S107 may further include the following steps S1071 to S1075:

[0153] Step S1071: Performing real-time vehicle-side calibration on the external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera according to the real-time point cloud data and the real-time image data;

[0154] Step S1072: Determine whether the real-time vehicle-side calibration of the external parameter is successful; if so, jump to step S1073; if not, jump to step S1074;

[0155] Step S1073: completing the repair of the second type of fault;

[0156] Step S1074: determine whether the number of real-time vehicle-side calibrations is greater than a preset number threshold; if so, jump to step S1075; if not, jump to step S1071;

[0157] Step S1075: Control the cloud server connected to the vehicle to send an updated calibration file to the vehicle to complete the repair of the second type of fault, wherein the updated calibration file includes the accurate internal parameters of the vehicle-mounted laser radar and the accurate external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera.

[0158] In this embodiment, when it is determined that the vehicle-mounted laser radar has a second type of fault, the external parameters of the vehicle-mounted laser radar can be first calibrated in real time on the vehicle side according to the real-time point cloud data collected by the vehicle-mounted laser radar and the real-time image data collected by the vehicle-mounted camera. When the real-time vehicle-side calibration of the external parameters is successful, the repair of the second type of fault is completed. When the external parameters are not successfully calibrated on the vehicle side in real time, the number of real-time vehicle-side calibrations can be determined. If it is less than or equal to the preset number threshold, the real-time vehicle-side calibration of the external parameters can be continued to be attempted; if it is greater than the number threshold, the cloud server connected to the vehicle communication can send an updated calibration file to the vehicle to complete the repair of the second type of fault. In other words, since the link between the vehicle and the cloud server is long, and communication with the cloud server requires traffic charges, when the vehicle-mounted laser radar is detected to have a second type of fault, the vehicle side can first attempt to perform real-time vehicle-side calibration on the external parameters of the vehicle-mounted laser radar. When the attempts are unsuccessful for many times, the cloud server can send an updated calibration file to update the internal and external parameters of the vehicle-mounted laser radar. Those skilled in the art can set the value of the number threshold according to the needs of actual applications.

[0159] In one embodiment, real-time vehicle-side calibration can be achieved through a preset model using real-time point cloud data and real-time image data obtained by collecting the same target in the same target collection area using a vehicle-mounted laser radar and a vehicle-mounted camera.

[0160] In one implementation, if the cloud server still cannot repair the second type of fault after sending an updated calibration file, the driver or relevant personnel will be notified to perform offline maintenance of the vehicle-mounted laser radar.

[0161] In one embodiment, please refer to the attached Figure 3 and Figure 4 , Figure 3is a schematic flow chart of the main steps of a method for diagnosing and processing a fault of a vehicle-mounted laser radar according to an implementation of an embodiment of the present invention. Figure 4 It is a main system structure block diagram of a system for implementing a fault diagnosis and processing method for a vehicle-mounted laser radar according to an implementation of an embodiment of the present invention.

[0162] like Figure 3 and Figure 4 As shown, in this embodiment, the fault diagnosis and processing method of the vehicle-mounted laser radar may include:

[0163] Step S201: Acquire real-time point cloud data collected by the vehicle-mounted laser radar, real-time image data collected by the vehicle-mounted camera, and real-time vehicle-side signals.

[0164] Step S202: The vehicle-mounted laser radar data analyzer performs fault analysis on the vehicle-mounted laser radar based on the real-time point cloud data collected by the vehicle-mounted laser radar, the real-time image data collected by the vehicle-mounted camera, and the real-time vehicle-side signal.

[0165] In this embodiment, the vehicle-mounted laser radar can be subjected to fault analysis based on the real-time point cloud data collected by the vehicle-mounted laser radar, the real-time image data collected by the vehicle-mounted camera, and the real-time vehicle-side signal to obtain the fault type of the vehicle-mounted laser radar, such as the first type of fault, the second type of fault, etc.

[0166] Step S203: Send the fault type analyzed by the vehicle-mounted laser radar data analyzer to the intelligent gateway, and send the fault type to the cloud server through the intelligent gateway.

[0167] Step S204: the cloud server clears or repairs the fault according to the fault type.

[0168] In this embodiment, the cloud server can clear or repair the fault according to the fault type. If the fault type is a first type fault, the cloud server can issue a fault alarm message to notify relevant personnel to repair the vehicle-mounted laser radar. If the fault type is a second type fault, the cloud server can issue an updated calibration file to enable the vehicle-mounted laser radar to obtain accurate internal and external parameters.

[0169] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0170] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0171] Furthermore, the present invention also provides a control device. In an embodiment of a control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the fault diagnosis and processing method of the vehicle-mounted laser radar of the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the fault diagnosis and processing method of the vehicle-mounted laser radar of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The control device can be a control device device formed by various electronic devices.

[0172] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the fault diagnosis and processing method of the vehicle-mounted laser radar of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the fault diagnosis and processing method of the above-mentioned vehicle-mounted laser radar. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0173] Furthermore, the present invention also provides a vehicle. In a vehicle embodiment according to the present invention, the vehicle may include the control device in the above control device embodiment.

[0174] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0175] Those skilled in the art will appreciate that the modules in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present invention.

[0176] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A fault diagnosis and processing method for a vehicle-mounted laser radar, characterized in that: include: Compare the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target; Determine whether the vehicle-mounted laser radar has a first type of fault according to the comparison result; Get the current scene where the vehicle is located; When the first type of fault exists, determining whether to handle the first type of fault according to the current scenario of the vehicle; When the first type of fault exists, judging whether to process the first type of fault according to the current scene of the vehicle includes: Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors; If it is determined that the current scene of the vehicle is not a preset scene, a fault alarm message is sent to the vehicle-mounted device; If it is determined that the current scene of the vehicle is a preset scene, no further processing is performed; or, Determining whether the current scene in which the vehicle is located is a preset scene, wherein the preset scene is a scene in which point cloud data is missing due to environmental factors; If it is determined that the current scene of the vehicle is not a preset scene, determining the fault duration of the first type of fault; If the fault duration exceeds a preset duration threshold, a fault alarm message is sent to the vehicle-mounted device; When the fault duration does not exceed the preset duration threshold, no further processing is performed; If it is determined that the current scene in which the vehicle is located is a preset scene, no further processing is performed.

2. The method according to claim 1, characterized in that The comparing the real-time point cloud data collected by the vehicle-mounted laser radar for the first preset target with the pre-stored standard point cloud data of the first preset target includes: Comparing the real-time point cloud data with the standard point cloud data to obtain the similarity between the real-time point cloud data and the standard point cloud data; The determining, according to the comparison result, whether the vehicle-mounted laser radar has a first type of fault includes: When the similarity is less than a preset similarity threshold, it is determined that the vehicle-mounted laser radar has the first type of fault.

3. The method according to claim 1, characterized in that The obtaining of the current scene of the vehicle includes: The real-time vehicle-side signal and the real-time image data collected by the vehicle-mounted camera are integrated in real time; The current scene of the vehicle is obtained according to the result of the real-time fusion.

4. The method according to claim 3, characterized in that The real-time vehicle-side signal includes at least one of a vehicle-side CAN signal, an RTK signal and an IMU signal.

5. The method according to claim 1, characterized in that Also includes: Acquire real-time point cloud data and real-time image data collected by the vehicle-mounted laser radar and the vehicle-mounted camera for a second preset target respectively; By comparing the real-time point cloud data and the real-time image data, determining whether the vehicle-mounted laser radar has a second type of fault; If it is determined that the vehicle-mounted laser radar has the second type of fault, the second type of fault is repaired.

6. The method according to claim 5, characterized in that The step of comparing the real-time point cloud data with the real-time image data to determine whether the vehicle-mounted laser radar has a second type of fault comprises: Acquire first position information of the second preset target according to the real-time point cloud data; Acquire second position information of the second preset target according to the real-time image data; Comparing the first position information with the second position information according to external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera to obtain a position deviation between the first position information and the second position information; If the position deviation is greater than a preset deviation threshold, it is determined that the vehicle-mounted laser radar has the second type of fault.

7. The method according to claim 5, characterized in that If it is determined that the vehicle-mounted laser radar has the second type of fault, repairing the second type of fault includes: Step S1: performing real-time vehicle-side calibration on the external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera according to the real-time point cloud data and the real-time image data; Step S2: Determine whether the real-time vehicle-side calibration of the external parameter is successful; if so, jump to step S3; if not, jump to step S4; Step S3: completing the repair of the second type of fault; Step S4: Determine whether the number of real-time vehicle-side calibrations is greater than a preset number threshold; if so, jump to step S5; if not, jump to step S1; Step S5: Control the cloud server connected to the vehicle to send an updated calibration file to the vehicle to complete the repair of the second type of fault, wherein the updated calibration file includes the accurate internal parameters of the vehicle-mounted laser radar and the accurate external parameters between the vehicle-mounted laser radar and the vehicle-mounted camera.

8. A control device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the fault diagnosis and processing method of the vehicle-mounted laser radar according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the fault diagnosis and processing method of the vehicle-mounted laser radar according to any one of claims 1 to 7.

10. A vehicle, characterized in that: The vehicle comprises the control device according to claim 8.

Images