Sensor foreign matter processing method and system

Abstract

The invention provides a sensor foreign matter processing method and system, and the method comprises the steps: obtaining sensor data collected by a sensor disposed on a vehicle, and carrying out the recognition of the sensor data through a preset recognition algorithm, so as to extract foreign matter data containing foreign matter information from the sensor data; analyzing the foreign matter data by using a preset foreign matter classification algorithm to obtain a foreign matter type on the sensor; the processing mode of the sensor is determined according to the foreign matter type, the sensor is processed through the processing mode, and the processing mode comprises cleaning or data compensation of the sensor. The foreign matter type can be accurately recognized to achieve efficient cleaning when the sensor needs to be cleaned, it can be guaranteed that the sensing capacity of the vehicle is not affected in the sensor cleaning process, and the risk of traffic accidents of the vehicle is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor cleaning, and in particular to a method and system for processing foreign matter in a sensor. Background Art

[0002] Sensors are an important component of autonomous vehicles. They support autonomous driving by collecting information around the vehicle, such as road conditions, obstacles, and the location of other vehicles. The sensing surface of the sensor can easily absorb foreign matter such as rain, snow, frost, and dust during use, causing sensor occlusion and thus affecting the vehicle's perception capabilities. Therefore, cleaning the sensor is crucial.

[0003] At present, the cleaning of sensors on autonomous vehicles mainly relies on regular manual cleaning, specifically using a cleaning cloth or a spray can to clean the sensors. However, in the actual cleaning process, manual cleaning often takes a lot of time and effort, and the cleaning efficiency is low. At the same time, since the sensors are located in different positions of the vehicle, some positions may be difficult to reach, and foreign matter on the sensors cannot be completely and thoroughly removed, resulting in poor cleaning effects. In addition, improper manual cleaning can easily cause scratches or damage to the sensor surface, which in turn has a serious impact on the performance of the sensor and may even cause the sensor to fail. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a method and system for handling foreign matter in sensors, aiming to solve the technical problem that in the prior art, the cleaning of sensors on autonomous driving vehicles mainly relies on regular manual cleaning, which has low cleaning efficiency and poor cleaning effect, and improper manual cleaning can easily cause scratches or damage to the sensor surface, which in turn has a serious impact on the performance of the sensor and may even cause the sensor to fail.

[0005] In one aspect, the present invention provides a method for processing foreign matter in a sensor, which is characterized by comprising:

[0006] Acquire sensor data collected by sensors arranged on the vehicle, and identify the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data;

[0007] Analyzing the foreign object data using a preset foreign object classification algorithm to obtain the type of foreign object on the sensor;

[0008] A processing method for the sensor is determined according to the type of the foreign matter, and the sensor is processed by the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

[0009] Compared with the prior art, the method for handling foreign matter on sensors of the present invention has the following beneficial effects: the type of foreign matter on the sensor is automatically identified through a preset recognition algorithm and a preset foreign matter classification algorithm, and a matching processing method is automatically confirmed to process the sensor according to the type of foreign matter. The processing method includes cleaning the sensor or performing data compensation on the sensor. It can not only monitor the actual status of the sensor in real time, and accurately identify the type of foreign matter when the sensor needs cleaning to achieve efficient cleaning and ensure the normal operation of the sensor, but also ensure that the vehicle's perception ability is not affected during the sensor cleaning process, thereby reducing the risk of traffic accidents involving the vehicle.

[0010] In addition, the sensor foreign body processing method according to the present invention may also have the following additional technical features:

[0011] Furthermore, the vehicle is provided with a cleaning assembly, the cleaning assembly comprising a cleaning liquid tank and an air storage tank, the cleaning liquid tank is used to store cleaning liquid, the air storage tank is used to store high-pressure gas, and the step of processing the sensor by the processing method comprises:

[0012] When the foreign matter is stubborn, the cleaning liquid in the cleaning liquid tank and the high-pressure gas in the air storage tank are controlled to perform spray cleaning.

[0013] Furthermore, the step of processing the sensor by the processing method includes:

[0014] When the foreign matter is a stain or an obstruction, the high-pressure gas in the air storage tank is controlled to perform spray cleaning.

[0015] Furthermore, the step of processing the sensor by the processing method includes:

[0016] When the type of the foreign matter is dust, the cleaning liquid in the cleaning liquid tank is controlled to perform flushing.

[0017] Furthermore, the cleaning component further comprises a heater, and the heater is used to heat the high-pressure gas. The step of processing the sensor by the processing method comprises:

[0018] When the type of the foreign matter is frost, the high-pressure gas in the gas storage tank and the heating unit are controlled to perform heating and cleaning.

[0019] Further, the sensor includes a plurality of sensors whose preset fields of view overlap each other, and the step of processing the sensors by the processing method includes:

[0020] When a data compensation condition is met, a nearby sensor is used to perform field of view compensation on the sensor that needs to be processed, wherein the data compensation condition includes any one of the sensor needing to be cleaned or the sensor being damaged.

[0021] Furthermore, after the step of acquiring sensor data collected by sensors arranged on the vehicle and identifying the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data, the step further includes:

[0022] According to the recognition result of the sensor data by the preset recognition algorithm, determine whether the sensor needs to be cleaned;

[0023] If the sensor data does not include foreign body data, it is determined that the sensor does not need to be cleaned, the sensor processing is terminated, and the vehicle operates normally;

[0024] If the sensor data includes foreign matter data, it is determined that the sensor needs to be cleaned, and a preset foreign matter classification algorithm is used to analyze the foreign matter data to obtain the type of foreign matter on the sensor.

[0025] Furthermore, after the step of determining the processing method of the sensor according to the type of the foreign matter and processing the sensor by the processing method, the step further includes:

[0026] The sensor data collected by the sensors deployed on the vehicle are obtained again, and the sensor data are identified using a preset identification algorithm to determine whether the sensor is clean;

[0027] If yes, the processing of the sensor ends and the vehicle operates normally;

[0028] If not, the process returns to executing the acquisition of sensor data collected by sensors arranged on the vehicle, and uses a preset recognition algorithm to recognize the sensor data, so as to extract foreign object data containing foreign object information from the sensor data.

[0029] Another aspect of the present invention provides a sensor foreign body processing system, comprising:

[0030] An acquisition module, used to acquire sensor data collected by sensors arranged on the vehicle, and identify the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data;

[0031] An analysis module, used to analyze the foreign object data using a preset foreign object classification algorithm to obtain the type of foreign object on the sensor;

[0032] A processing module is used to determine a processing method of the sensor according to the type of the foreign matter, and process the sensor by the processing method, wherein the processing method includes cleaning the sensor or performing data compensation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the field of view of the sensor arranged on the vehicle in the first embodiment of the present invention;

[0034] Figure 2 is a flow chart of a method for handling foreign matter in a sensor in a first embodiment of the present invention;

[0035] Figure 3 is a flow chart of a method for handling foreign matter in a sensor according to a second embodiment of the present invention;

[0036] Figure 4 A block diagram of the sensor foreign body processing system provided by the present invention.

[0037] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0038] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0039] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0041] In order to solve the technical problem that in the prior art, the cleaning of sensors on autonomous driving vehicles mainly relies on regular manual cleaning, which has low cleaning efficiency and poor cleaning effect, and improper manual cleaning can easily cause scratches or damage to the sensor surface, which in turn has a serious impact on the performance of the sensor and may even cause the sensor to fail, the present application provides a sensor foreign body processing method and system, which automatically identifies the type of foreign matter on the sensor through a preset recognition algorithm and a preset foreign matter classification algorithm, and automatically confirms a matching processing method to process the sensor according to the type of foreign matter. The processing method includes cleaning the sensor or performing data compensation on the sensor. It can not only monitor the actual status of the sensor in real time, and accurately identify the type of foreign matter when the sensor needs cleaning to achieve efficient cleaning and ensure the normal operation of the sensor, but also ensure that the vehicle's perception ability is not affected during the sensor cleaning process, thereby reducing the risk of traffic accidents involving the vehicle.

[0042] In order to facilitate understanding of the present invention, the technical solution of the present application will be described in detail with specific embodiments below. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0043] Embodiment 1

[0044] In order to realize the vehicle's perception of the surrounding environment, the vehicle is generally equipped with multiple sensors to collect information around the vehicle. Figure 1 As shown, the sensors deployed on the vehicle include a wide-angle camera, a telephoto camera, a left main lidar, a right main lidar, a left blind spot radar and a right blind spot radar. The preset fields of view of these sensors can partially or completely overlap with each other. It can be understood that the preset field of view range of one or several sensors has an area that partially or completely overlaps with the preset field of view range of the remaining sensors. In this way, when one of the sensors needs to be cleaned or is damaged, the data in the field of view can be compensated by nearby sensors to ensure that the vehicle's perception ability is not affected.

[0045] See also Figure 2 , which is a flow chart of a method for processing foreign matter in a sensor in the first embodiment of the present application, the method in this embodiment may include steps S101 to S103:

[0046] S101, acquiring sensor data collected by sensors arranged on a vehicle, and identifying the sensor data using a preset identification algorithm to extract foreign object data containing foreign object information from the sensor data.

[0047] The sensors deployed on the vehicle are detected in real time, and the sensor data collected by the sensors are obtained, wherein the sensors include a wide-angle camera, a telephoto camera, a left main laser radar, and a right main laser radar. The obtained sensor data is identified using a preset recognition algorithm, wherein the preset recognition algorithm can be obtained by training and learning a large amount of standard sensor data to understand the features in the sensor data and identify the occlusion of the sensor in real time. At the same time, by combining the real-time data of sensors such as the wide-angle camera, the telephoto camera, the left main laser radar, and the right main laser radar, the accuracy and robustness of the recognition can be improved. The specific training method is a technical means that can be understood by those skilled in the art, and will not be repeated here. In actual use, the preset recognition algorithm identifies the obtained sensor data based on the preset standard sensor data. If there is a foreign object on the sensor, it can extract the foreign object data containing the foreign object information from the obtained sensor data.

[0048] S102, analyzing the foreign body data using a preset foreign body classification algorithm to obtain the type of foreign body on the sensor;

[0049] In this embodiment, the preset foreign object classification algorithm can be obtained by training and learning a large amount of foreign object data or using a preset neural network to identify and classify different types of foreign objects. Such analysis can help determine the cleaning method of the sensor. The specific training method and the selection of the neural network are technical means that can be understood by those skilled in the art and will not be repeated here. In actual use, the preset foreign object classification algorithm can accurately identify the type of foreign object on the sensor based on the features in the foreign object data, where the foreign object type includes any one of stubborn foreign objects, stains or obstructions, dust, and frost.

[0050] S103, determining a processing method for the sensor according to the type of the foreign matter, and processing the sensor using the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

[0051] In summary, a sensor foreign body processing method of the present application automatically identifies the type of foreign body on the sensor through a preset recognition algorithm and a preset foreign body classification algorithm, and automatically confirms a matching processing method to process the sensor according to the type of foreign body. The processing method includes cleaning the sensor or performing data compensation on the sensor. It can not only monitor the actual status of the sensor in real time, and accurately identify the type of foreign body when the sensor needs cleaning to achieve efficient cleaning and ensure the normal operation of the sensor, but also ensure that the vehicle's perception ability is not affected during the sensor cleaning process, thereby reducing the risk of traffic accidents involving the vehicle.

[0052] Embodiment 2

[0053] See also Figure 3, which is a flow chart of a method for processing foreign matter in a sensor in a second embodiment of the present application, the method in this embodiment may include steps S201 to S205:

[0054] S201, acquiring sensor data collected by sensors arranged on the vehicle, and identifying the sensor data using a preset identification algorithm to extract foreign object data containing foreign object information from the sensor data.

[0055] The sensors deployed on the vehicle are detected in real time, and the sensor data collected by the sensors are obtained, wherein the sensors include a wide-angle camera, a telephoto camera, a left main laser radar, and a right main laser radar. The obtained sensor data is identified using a preset recognition algorithm, wherein the preset recognition algorithm can be obtained by training and learning a large amount of standard sensor data to understand the features in the sensor data and identify the occlusion of the sensor in real time. At the same time, by combining the real-time data of sensors such as the wide-angle camera, the telephoto camera, the left main laser radar, and the right main laser radar, the accuracy and robustness of the recognition can be improved. The specific training method is a technical means that can be understood by those skilled in the art, and will not be repeated here. In actual use, the preset recognition algorithm identifies the obtained sensor data based on the preset standard sensor data. If there is a foreign object on the sensor, it can extract the foreign object data containing the foreign object information from the obtained sensor data.

[0056] S202: Determine whether the sensor needs to be cleaned according to the recognition result of the sensor data by a preset recognition algorithm.

[0057] If the sensor data does not include foreign matter data, it is determined that the sensor does not need to be cleaned, the processing of the sensor is terminated, and the vehicle operates normally.

[0058] If the sensor data includes foreign matter data, it is determined that the sensor needs to be cleaned, and step S203 is executed.

[0059] S203, analyzing the foreign matter data using a preset foreign matter classification algorithm to obtain the type of foreign matter on the sensor;

[0060] In this embodiment, the preset foreign object classification algorithm can be obtained by training and learning a large amount of foreign object data or using a preset neural network to identify and classify different types of foreign objects. Such analysis can help determine the cleaning method of the sensor. The specific training method and the selection of the neural network are technical means that can be understood by those skilled in the art and will not be repeated here. In actual use, the preset foreign object classification algorithm can accurately identify the type of foreign object on the sensor based on the features in the foreign object data, where the foreign object type includes any one of stubborn foreign objects, stains or obstructions, dust, and frost.

[0061] S204: Determine a processing method for the sensor according to the type of foreign matter, and process the sensor using the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

[0062] Specifically, the step of processing the sensor by the processing method includes step S2041-step S2042:

[0063] S2041. When a data compensation condition is met, use a nearby sensor to perform field of view compensation on the sensor that needs to be processed, wherein the data compensation condition includes any one of the sensor needing to be cleaned or the sensor being damaged.

[0064] In this embodiment, the method of field of view compensation may be:

[0065] When it is determined that the telephoto camera needs to be cleaned, the wide-angle camera, the left main lidar, and the right main lidar are used to cover the field of view of the telephoto camera.

[0066] When it is determined that the wide-angle camera needs to be cleaned, the telephoto camera, the left main lidar, and the right main lidar are used to cover the field of view of the wide-angle camera.

[0067] When it is determined that the left main LiDAR needs to be cleaned, the wide-angle camera, telephoto camera and right main LiDAR are used to cover the field of view of the left main LiDAR.

[0068] When it is determined that the right main LiDAR needs to be cleaned, the wide-angle camera, telephoto camera and left main LiDAR are used to cover the field of view of the right main LiDAR.

[0069] S2042. Clean the sensor using a cleaning component.

[0070] The vehicle is provided with a cleaning component, which includes a cleaning liquid tank, an air tank and a heater. The cleaning liquid tank is used to store cleaning liquid, the air tank is used to store high-pressure gas, and the heater is connected to the air tank to heat the high-pressure gas. As a specific example, in this embodiment, the vehicle is also provided with a controller, and the cleaning component also includes a water pump connected to the cleaning liquid tank, a water pump relay arranged between the water pump and the controller, an air pump connected to the air tank, an air pump relay arranged between the air pump and the controller, a first solenoid valve arranged between the water pump and the sensor, a second solenoid valve arranged between the air tank and the sensor, and a one-way valve arranged between the first solenoid valve and the water pump. In the actual cleaning process, the controller controls the water pump through the water pump relay to extract cleaning liquid from the cleaning liquid tank and spray it onto the sensor for cleaning, or the controller controls the air pump through the air pump relay to spray the high-pressure gas in the air tank onto the sensor for flushing.

[0071] Specifically, cleaning the sensor using the cleaning component includes:

[0072] When the foreign matter type is stubborn foreign matter, the cleaning liquid in the cleaning liquid tank and the high-pressure gas in the air storage tank are controlled to perform spray cleaning.

[0073] When the foreign matter is a stain or an obstruction, the high-pressure gas in the gas storage tank is controlled to perform jet cleaning.

[0074] When the type of foreign matter is dust, the cleaning liquid in the cleaning liquid tank is controlled to flush.

[0075] When the type of foreign matter is frost, the high-pressure gas in the gas storage tank and the heating unit are controlled to perform heating and cleaning.

[0076] S205, acquiring sensor data collected by sensors deployed on the vehicle again, and identifying the sensor data using a preset identification algorithm to determine whether the sensors are clean.

[0077] If yes, the processing of the sensor ends and the vehicle operates normally;

[0078] If not, return to step S201.

[0079] See also Figure 4 The present invention also provides a sensor foreign body processing system, the processing system 300 comprises:

[0080] The acquisition module 301 is used to acquire sensor data collected by sensors arranged on the vehicle, and identify the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data;

[0081] The analysis module 302 is used to analyze the foreign body data using a preset foreign body classification algorithm to obtain the type of foreign body on the sensor;

[0082] The processing module 303 is used to determine a processing method of the sensor according to the type of foreign matter, and process the sensor using the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

[0083] In summary, a sensor foreign body processing method and system of the present application automatically identifies the type of foreign body on the sensor through a preset recognition algorithm and a preset foreign body classification algorithm, and automatically confirms a matching processing method to process the sensor according to the type of foreign body. The processing method includes cleaning the sensor or performing data compensation on the sensor. It can not only monitor the actual status of the sensor in real time, and accurately identify the type of foreign body when the sensor needs cleaning to achieve efficient cleaning and ensure the normal operation of the sensor, but also ensure that the vehicle's perception ability is not affected during the sensor cleaning process, thereby reducing the risk of traffic accidents involving the vehicle.

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0085] The above embodiments only express several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of application of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention application shall be subject to the attached claims.

Claims

1. A method for handling foreign matter in a sensor, characterized in that: include: Acquire sensor data collected by sensors arranged on the vehicle, and identify the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data; Analyzing the foreign object data using a preset foreign object classification algorithm to obtain the type of foreign object on the sensor; A processing method for the sensor is determined according to the type of the foreign matter, and the sensor is processed by the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

2. The method for handling foreign matter in a sensor according to claim 1, characterized in that: The vehicle is provided with a cleaning assembly, the cleaning assembly comprising a cleaning liquid tank and an air storage tank, the cleaning liquid tank is used to store cleaning liquid, the air storage tank is used to store high-pressure gas, and the step of processing the sensor by the processing method comprises: When the foreign matter is stubborn, the cleaning liquid in the cleaning liquid tank and the high-pressure gas in the air storage tank are controlled to perform spray cleaning.

3. The method for processing foreign matter in a sensor according to claim 2, characterized in that: The step of processing the sensor by the processing method comprises: When the foreign matter is a stain or an obstruction, the high-pressure gas in the air storage tank is controlled to perform spray cleaning.

4. The method for processing foreign matter in a sensor according to claim 2, characterized in that: The step of processing the sensor by the processing method comprises: When the type of the foreign matter is dust, the cleaning liquid in the cleaning liquid tank is controlled to perform flushing.

5. The method for processing foreign matter in a sensor according to claim 2, characterized in that: The cleaning component further includes a heater, and the heater is used to heat the high-pressure gas. The step of processing the sensor by the processing method includes: When the type of the foreign matter is frost, the high-pressure gas in the gas storage tank and the heating unit are controlled to perform heating and cleaning.

6. The method for processing foreign matter in a sensor according to claim 1, characterized in that: The sensor includes a plurality of sensors whose preset fields of view overlap each other, and the step of processing the sensors by the processing method includes: When a data compensation condition is met, a nearby sensor is used to perform field of view compensation on the sensor that needs to be processed, wherein the data compensation condition includes any one of the sensor needing to be cleaned or the sensor being damaged.

7. The method for processing foreign matter in a sensor according to claim 1, characterized in that: The step of acquiring sensor data collected by sensors arranged on the vehicle and identifying the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data further includes: According to the recognition result of the sensor data by the preset recognition algorithm, determine whether the sensor needs to be cleaned; If the sensor data does not include foreign body data, it is determined that the sensor does not need to be cleaned, the processing of the sensor is terminated, and the vehicle operates normally; If the sensor data includes foreign matter data, it is determined that the sensor needs to be cleaned, and a preset foreign matter classification algorithm is used to analyze the foreign matter data to obtain the type of foreign matter on the sensor.

8. The method for processing foreign matter in a sensor according to claim 1, characterized in that: After the step of determining the processing method of the sensor according to the type of the foreign matter and processing the sensor by the processing method, the step further includes: The sensor data collected by the sensors deployed on the vehicle are obtained again, and the sensor data are identified using a preset identification algorithm to determine whether the sensor is clean; If yes, the processing of the sensor ends and the vehicle operates normally; If not, the process returns to executing the acquisition of sensor data collected by sensors arranged on the vehicle, and uses a preset recognition algorithm to recognize the sensor data, so as to extract foreign object data containing foreign object information from the sensor data.

9. A sensor foreign body processing system, characterized in that: include: An acquisition module, used to acquire sensor data collected by sensors arranged on the vehicle, and identify the sensor data using a preset recognition algorithm to extract foreign object data containing foreign object information from the sensor data; An analysis module, used to analyze the foreign object data using a preset foreign object classification algorithm to obtain the type of foreign object on the sensor; A processing module is used to determine a processing method of the sensor according to the type of the foreign matter, and process the sensor by the processing method, wherein the processing method includes cleaning the sensor or performing data compensation.

Images