Method and device for cleaning shelter on front window of vehicle and vehicle

Abstract

The invention is suitable for the technical field of shelter cleaning, and provides a vehicle front window shelter cleaning method and device and a vehicle, and the method comprises the steps: recognizing whether there is a shelter on a vehicle front window according to the obtained vehicle front window multi-mode information, and obtaining a shelter recognition result; and when the shielding object recognition result is that a shielding object exists, the type of the shielding object is recognized, the shielding object type is obtained, the vehicle front window multi-modal information comprises front window pressure information, radar point cloud information and / or a front window image, a target cleaning device is determined according to the shielding object type and the shielding area, obtained in advance, of the vehicle front window, and the target cleaning device is used for cleaning the vehicle front window. And controlling the target cleaning device according to the cleaning operation parameters and the cleaning operation parameters of the target cleaning device, and cleaning the shelter according to the cleaning operation parameters. The problems that in the prior art, the vehicle front window shielding object cleaning mode is single, and shielding objects cannot be rapidly removed can be solved.

Description

Technical Field

[0001] This application belongs to the field of obstruction removal technology, and in particular relates to a method, device and vehicle for removing obstructions from the windshield of a vehicle. Background Technology

[0002] The cleanliness of a vehicle's windshield directly affects the driver's visibility and the sensitivity of sensors and other equipment relied upon by technologies such as autonomous driving. Timely identification and removal of obstructions from the windshield helps maintain its cleanliness, ensuring safe driving even in complex road conditions.

[0003] Currently, the traditional method for identifying and clearing obstructions from vehicle windshields mainly relies on sensors at the front of the vehicle to identify the obstructions and then using traditional windshield wipers to spray water and remove them. However, this traditional method has a low accuracy rate in identifying obstructions, uses a single method, and cannot quickly remove obstructions, easily creating safety hazards and increasing the risk of rear-end collisions and other traffic accidents. Summary of the Invention

[0004] This application provides a method, device, and vehicle for cleaning obstructions from the windshield of a vehicle, which can solve the problem that the existing methods for cleaning obstructions from the windshield of a vehicle are limited and cannot quickly remove obstructions.

[0005] In a first aspect, embodiments of this application provide a method for clearing obstructions from a vehicle's windshield, including: Based on the acquired multimodal information of the vehicle's front window, the presence of obstructions on the vehicle's front window is identified to obtain an obstruction identification result; and when the obstruction identification result indicates the presence of an obstruction, the type of the obstruction is identified to obtain an obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information, and / or front window image; Based on the type of obstruction and the obstruction area of ​​the vehicle's windshield, a target cleaning device and its cleaning operation parameters are determined; wherein, the target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device; Control the target cleaning device to clean the obstruction according to the cleaning operation parameters.

[0006] Based on the acquired multimodal information of the vehicle's windshield, the presence of obstructions on the windshield is identified, yielding an obstruction identification result. Furthermore, when the obstruction identification result indicates the presence of an obstruction, the type of obstruction is identified, resulting in an obstruction category. Since the multimodal information of the vehicle's windshield includes windshield pressure information, radar point cloud information, and / or windshield images, determining the obstruction identification result based on this information is more accurate than relying solely on a single sensor or other single device. Moreover, when an obstruction is present, the category of the obstruction determined based on the multimodal information of the vehicle's windshield is also more accurate. Based on the type of obstruction and the area obstructed by the vehicle's windshield, a target cleaning device and its cleaning operation parameters are determined. The target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device. This ensures that the determined cleaning operation parameters of the target cleaning device are matched with the type and area of ​​the obstruction. By controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters, the obstruction on the vehicle's windshield can be quickly removed, ensuring safe driving. This effectively solves the problem that existing technologies have a single method for cleaning obstructions on vehicle windshields and cannot quickly remove obstructions.

[0007] In one possible implementation of the first aspect, the step of identifying whether there is an obstruction on the vehicle's windshield based on the acquired multimodal information of the vehicle's windshield to obtain an obstruction identification result; and, when the obstruction identification result indicates the presence of an obstruction, identifying the type of the obstruction to obtain an obstruction category, including: Based on the multimodal information of the vehicle's windshield, an initial identification of whether an obstruction exists on the vehicle's windshield is performed, resulting in an initial identification result of the obstruction. This initial identification result includes a first identification result, a second identification result, and / or a third identification result. The first identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on the windshield pressure information. The second identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on the windshield image. The third identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on the radar point cloud information. Based on the initial obstruction identification result, a comprehensive judgment is made on whether there is an obstruction on the vehicle's windshield to obtain the obstruction identification result; When the obstruction identification result indicates the presence of an obstruction, the type of the obstruction is comprehensively judged based on the initial obstruction category to obtain the obstruction category; wherein, the initial obstruction category includes an obstruction first category, an obstruction second category, and / or an obstruction third category; the obstruction first category is obtained by identifying the type of the obstruction based on the front window pressure information; the obstruction second category is obtained by identifying the type of the obstruction based on the front window image; the obstruction third category is obtained by identifying the type of the obstruction based on the radar point cloud information.

[0008] When identifying whether there is an obstruction on the vehicle's windshield based on the acquired multimodal information of the windshield, an initial obstruction identification result is obtained. This initial obstruction identification result includes a first identification result, a second identification result, and / or a third identification result. Specifically, the first identification result is obtained when identifying whether there is an obstruction on the vehicle's windshield based on windshield pressure information; the second identification result is obtained when identifying whether there is an obstruction on the vehicle's windshield based on windshield images; and the third identification result is obtained when identifying whether there is an obstruction on the vehicle's windshield based on radar point cloud information. Finally, a comprehensive assessment of whether there is an obstruction on the vehicle's windshield is performed based on the initial obstruction identification results. The judgment process yields an object identification result. By combining the first, second, and third identification results, the object identification result becomes more accurate. Furthermore, when the object identification result indicates the presence of an object, the type of the object is comprehensively judged based on the initial object category, which includes the first, second, and / or third object category, to obtain the object category. Because the first, second, and third object categories are combined, the final determined object category is also more accurate, thereby improving the accuracy of the object identification result. At the same time, when determining the presence of an object, the accuracy rate of object category identification is also improved.

[0009] In one possible implementation of the first aspect, the method further includes controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters, and then the method further includes: The temperature difference of the glass surface of the vehicle's windshield after cleaning is obtained; wherein, the temperature difference of the glass surface is used to characterize the temperature difference between the obstructed area and the unobstructed area of ​​the vehicle's windshield; the obstructed area is determined based on the location information of the obstruction. Based on the temperature difference on the glass surface, determine whether the obstruction has been completely removed; When it is determined that the obstruction has not been completely cleared, the target cleaning device is controlled to clean the obstruction again according to the cleaning operation parameters.

[0010] The control target cleaning device cleans the obstruction according to the cleaning operation parameters. After that, the temperature difference of the vehicle's windshield surface is obtained. The temperature difference of the glass surface is used to characterize the temperature difference between the obstructed area and the unobstructed area of ​​the vehicle's windshield. The obstructed area is determined based on the location information of the obstruction. The temperature difference of the glass surface can be used to determine whether the obstruction has been completely cleaned. When it is determined that the obstruction has not been completely cleaned, the control target cleaning device cleans the obstruction again according to the cleaning operation parameters. This can save resources and also clean the obstruction again when it is difficult to clean, thereby improving the cleaning effect and keeping the vehicle's windshield clean, ensuring the safe driving of the vehicle.

[0011] In one possible implementation of the first aspect, determining the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield includes: When the type of obstruction is rain or snow, the target cleaning device is determined to be a windshield wiper. When the type of obstruction is not rain or snow, and the obstruction area is less than a first preset threshold, the target cleaning device is determined to be a nitrogen injector. When the type of obstruction is not rain or snow, and the obstruction area is greater than or equal to the first preset threshold, the target cleaning device is determined to be a laser emitting device.

[0012] When the obstruction is classified as rain or snow, the target cleaning device is determined to be the windshield wiper. When the obstruction is not classified as rain or snow and the obstruction area is less than a first preset threshold, the target cleaning device is determined to be the nitrogen injector. When the obstruction is not classified as rain or snow and the obstruction area is greater than or equal to the first preset threshold, the target cleaning device is determined to be the laser emitter. This allows for the determination of a matching target cleaning device based on the type of obstruction and the size of the obstruction area. This enables the use of a cleaning device that matches the type and size of the obstruction to clean obstructions on the vehicle's windshield, improving cleaning efficiency and effectiveness, facilitating rapid removal of obstructions, and ensuring safe vehicle operation.

[0013] In one possible implementation of the first aspect, when the target cleaning device is determined to be a nitrogen injector, the cleaning operation parameters of the target cleaning device are determined, including: Based on the type of obstruction, the target injection pressure, target injection time, and target injection mode of the nitrogen injector are determined respectively.

[0014] When the target cleaning device is determined to be a nitrogen injector, the cleaning operation parameters of the nitrogen injector can be determined according to the type of obstruction, including the target injection pressure, target injection time, and target injection mode. This ensures that the target injection pressure, target injection time, and target injection mode are matched with the type of obstruction, thereby controlling the nitrogen injector to spray and clean the obstruction according to the target injection pressure, target injection time, and target injection mode, improving the spray cleaning effect, and enabling the obstruction on the vehicle's windshield to be quickly and cleaned.

[0015] In one possible implementation of the first aspect, when the occlusion identification result indicates the presence of an occlusion, the method further includes: Based on the front window image, determine the position information of the obstruction; Based on the location information, the target angle of the target cleaning device is determined; Control the target cleaning device to clean the obstruction according to the target angle.

[0016] When the obstruction identification result indicates the presence of an obstruction, the location information of the obstruction is determined based on the front window image. Based on the location information, the target angle of the target cleaning device is determined, which makes the target angle more closely match the location information of the obstruction. The target cleaning device is then controlled to clean the obstruction according to the target angle, which helps to achieve rapid cleaning of obstructions on the vehicle's front window.

[0017] In one possible implementation of the first aspect, the method further includes: When the obstruction area is greater than or equal to the second preset threshold and less than the first preset threshold, the vehicle's front window visible area image, rearview mirror image and blind spot monitoring image are acquired. The front window visible area image, the rearview mirror image, and the blind spot monitoring image are aligned and stitched together to obtain a virtual front view image; The virtual front view image is projected onto the vehicle's dashboard area for display.

[0018] When the obstructed area is greater than or equal to a second preset threshold and less than a first preset threshold, images of the vehicle's front window visible area, rearview mirror, and blind spot monitoring are acquired. These images are then aligned and stitched together to obtain a virtual front view image. This virtual front view image contains image information from the vehicle's front window visible area, the rearview mirror, and the blind spot monitoring. The virtual front view image is then projected onto the vehicle's dashboard for display. When the vehicle's front window is largely obstructed, the stitched virtual image displays various information, allowing the driver to assess the surrounding environment and improve driving safety.

[0019] In one possible implementation of the first aspect, the method further includes: When the obstruction area is greater than or equal to the first preset threshold, an emergency avoidance signal is sent to vehicles behind.

[0020] When the obstruction area is greater than or equal to the first preset threshold, an emergency avoidance signal is sent to vehicles behind. This can promptly alert vehicles behind when the windshield of a vehicle is largely obstructed, reducing the risk of rear-end collisions and helping to ensure safe driving.

[0021] Secondly, embodiments of this application provide a device for clearing obstructions from a vehicle's windshield, comprising: An obstruction recognition module is used to identify whether there is an obstruction on the vehicle's front window based on the acquired multimodal information of the vehicle's front window, and obtain an obstruction recognition result; and when the obstruction recognition result indicates the presence of an obstruction, to identify the type of the obstruction and obtain an obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information and / or front window image; The cleaning parameter determination module is used to determine the target cleaning device and the cleaning operation parameters of the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield; wherein the target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device; The control module is used to control the target cleaning device to clean the obstruction according to the cleaning operation parameters.

[0022] Thirdly, embodiments of this application provide a device for clearing obstructions from a vehicle's windshield, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the above-mentioned embodiments.

[0023] Fourthly, embodiments of this application provide a vehicle, including a device for clearing obstructions from the vehicle's windshield; wherein the device for clearing obstructions from the vehicle's windshield is used to perform the method described in any of the preceding claims.

[0024] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0025] Sixthly, embodiments of this application provide a computer program product that, when run on a vehicle windshield obstruction removal device, causes the vehicle windshield obstruction removal device to perform any of the methods described above.

[0026] The beneficial effects of this application embodiment compared with the prior art are as follows: By identifying whether there is an obstruction on the vehicle's front window based on the acquired multimodal information of the vehicle's front window, an obstruction identification result is obtained; and when the obstruction identification result indicates the presence of an obstruction, the type of obstruction is identified to obtain the obstruction category. Since the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information, and / or front window image, determining the obstruction identification result based on the multimodal information of the vehicle's front window is more accurate than relying on a single sensor for identification, and when an obstruction is present... In this way, the obtained obstruction categories are more accurate. Furthermore, based on the obstruction category and the pre-observed obstruction area of ​​the vehicle's windshield, the target cleaning device and its cleaning operation parameters are determined. This ensures that the determined cleaning operation parameters of the target cleaning device match the obstruction category and obstruction area. By controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters, the obstruction of the vehicle's windshield can be quickly removed, ensuring the safe driving of the vehicle. This effectively solves the problem that the existing technology has a single method for cleaning obstructions on the vehicle's windshield and cannot quickly remove obstructions. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a method for cleaning obstructions from the windshield of a vehicle, provided in one embodiment of this application.

[0029] Figure 2 This is a schematic diagram of a vehicle windshield obstruction removal device provided in one embodiment of this application. Detailed Implementation

[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0032] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0033] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0034] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0036] In existing technology, when a vehicle's windshield is obstructed by an obstacle, the windshield wipers are typically used to clear the obstruction, keeping the windshield clean and providing the driver with a clear view, thus ensuring safe driving. However, these traditional windshield wipers cannot quickly remove obstacles such as birds and plastic bags, resulting in poor cleaning performance and potentially affecting the driver's visibility, thereby compromising vehicle safety.

[0037] This application provides a method for clearing obstructions from a vehicle's windshield. See [link to relevant documentation]. Figure 1 , Figure 1This is a flowchart illustrating a method for clearing obstructions from a vehicle's windshield according to an embodiment of this application, comprising: Step S11: Based on the acquired multimodal information of the vehicle's front window, identify whether there is an obstruction on the vehicle's front window to obtain an obstruction identification result; and when the obstruction identification result indicates that an obstruction exists, identify the type of obstruction to obtain the obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information and / or front window image.

[0038] Step S12: Determine the target cleaning device and the cleaning operation parameters of the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield; wherein, the target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device.

[0039] Step S13: Control the target cleaning device to clean the obstruction according to the cleaning operation parameters.

[0040] Specifically, radar arrays are typically installed on both sides of the vehicle's roof. These arrays can be composed of multiple lidar units. The radar arrays monitor the environment around the vehicle, generating radar point cloud information. This radar point cloud information allows for real-time monitoring of the trajectory of objects flying within 10 meters in front of the vehicle. Based on these trajectories, the presence of obstructions on the vehicle's windshield can be predicted, along with the area of ​​obstruction if present. The radar point cloud information can be a three-dimensional point cloud map of the vehicle's surroundings, used to determine the distance, orientation, height, and shape of obstacles near the windshield, and can also include the trajectory of these obstacles.

[0041] The windshield of a vehicle can be made of functional laminated glass, such as by embedding a nano-pressure-sensitive film into the glass interlayer of the windshield, which can acquire real-time windshield pressure information. This windshield pressure information can be used to characterize the pressure at multiple locations on the windshield, or to characterize the average pressure across multiple areas of the windshield. The windshield pressure information can also be used to identify whether there are obstructions on the windshield, and the type of obstruction.

[0042] A camera can also be installed at the location of the vehicle's interior rearview mirror. This camera can be an infrared camera, a regular camera, or a surround-view camera. When the camera is an infrared camera, the real-time image of the windshield acquired by the infrared camera is an infrared windshield image. This infrared windshield image can be used to detect the long-wave infrared energy radiated by objects in the windshield and generate a temperature distribution map. When the camera is a regular camera, the real-time image of the windshield acquired by the regular camera is a visible light windshield image. The visible light windshield image can be used to generate image features using deep learning technology to identify information about the vehicle's windshield. When the camera is a surround-view camera, the real-time image of the windshield acquired by the surround-view camera is a panoramic windshield image. The panoramic windshield image can also be used to generate image features using deep learning technology to identify information about the vehicle's windshield, including obstacle detection.

[0043] The aforementioned radar array, functional laminated glass, and cameras can be used to acquire multimodal information about the vehicle's windshield. This multimodal information includes windshield pressure information, radar point cloud information, and / or windshield images. Windshield pressure information, radar point cloud information, and windshield images can all be acquired in real time, and historical data on these parameters can be saved.

[0044] Based on any combination of the acquired windshield pressure information, radar point cloud information, and windshield image, the presence of obstructions on the vehicle's windshield can be identified, yielding an obstruction identification result. This result indicates either the presence or absence of an obstruction. When the obstruction identification result confirms the presence of an obstruction on the vehicle's windshield, the type of obstruction can also be identified based on any combination of the windshield pressure information, radar point cloud information, and windshield image, resulting in an obstruction category. This category can include rain, snow, leaves, plastic bags, or living organisms, among other things.

[0045] For example, radar point cloud information can be used to identify whether there is an obstruction on the vehicle's windshield, obtaining the probability of its presence. If the probability of an obstruction is greater than a preset probability threshold, the obstruction is identified as present. When the obstruction is determined to be present on the vehicle's windshield, the type of obstruction can also be identified based on the radar point cloud information, obtaining the obstruction category. If the probability of an obstruction is not greater than a preset probability threshold, the obstruction is identified as not present.

[0046] In another optional example, the presence of an obstruction on the vehicle's windshield can be identified based on any two of the windshield pressure information, radar point cloud information, and windshield image, yielding an obstruction identification result. When the obstruction identification result confirms the presence of an obstruction on the vehicle's windshield, the type of obstruction can also be identified based on any two of the windshield pressure information, radar point cloud information, and windshield image, yielding an obstruction category. For example, a comprehensive identification of the presence of an obstruction on the vehicle's windshield can be performed based on both windshield pressure information and the windshield image, yielding an obstruction identification result. When the obstruction identification result confirms the presence of an obstruction on the vehicle's windshield, the type of obstruction can also be identified based on both windshield pressure information and the windshield image, yielding an obstruction category.

[0047] It should be noted that the presence of obstructions on the vehicle's windshield can also be comprehensively identified and judged based on windshield pressure information, radar point cloud information, and windshield images to obtain obstruction identification results. When the obstruction identification result is determined to be an obstruction on the vehicle's windshield, the type of obstruction is comprehensively identified and judged based on windshield pressure information, radar point cloud information, and windshield images to obtain the obstruction category.

[0048] When the system determines that an obstruction exists, it can also issue a warning to remind the driver that there is an obstruction on the vehicle's windshield and ask the driver to take timely measures to remove the obstruction.

[0049] In another optional example, when the obstruction identification result indicates the presence of an obstruction, the obstruction area of ​​the vehicle's windshield can be determined based on at least one of the following: windshield pressure information, radar point cloud information, and windshield image. Here, the obstruction area of ​​the vehicle's windshield represents the area obstructed by the obstruction on the windshield. The obstruction ratio can also be calculated based on the total area of ​​the vehicle's windshield and the obstruction area of ​​the windshield.

[0050] It should be noted that the occlusion area of ​​the vehicle's windshield can be determined using deep learning technology based on at least one of the following: windshield pressure information, radar point cloud information, and windshield image. This embodiment does not specifically limit the determination method. For example, features can be extracted from the windshield image to obtain the shadow area of ​​the obstruction, and then the occlusion area can be determined based on the shadow area.

[0051] Because different types of obstructions and their varying sizes have different impacts on the driver's visibility, the most efficient removal methods also differ. The target removal method can be determined based on the type of obstruction and the area obstructed by the vehicle's windshield. The removal method can be a tiered response execution method, without a specific limit on the number of tiers. For example, tiered response execution methods include Level 1, Level 2, and Level 3 response execution methods. The target removal method, on the other hand, is a tiered response execution method matched to the type of obstruction and the area obstructed by the vehicle's windshield, enabling rapid removal of the obstruction. The target removal method can include a target removal device and its cleaning operation parameters. When the target removal device is a nitrogen injector, the cleaning operation parameters can include the target injection angle, target injection pressure, target injection time, and target injection mode.

[0052] It's worth noting that the vehicle is equipped with windshield wipers, which clear rain, snow, dust, and other obstructions from the windshield, ensuring the driver has a clear view in various weather and road conditions, thus improving driving safety. The wiper blades can also house nitrogen injectors, which spray nitrogen gas to remove obstacles such as plastic bags and leaves from the windshield. These nitrogen injectors can be high-pressure nitrogen injectors, which can continuously spray nitrogen gas to remove obstacles from the windshield more quickly. The vehicle also features a laser emitter that can pulse-remove stubborn deposits from the windshield. For example, the laser emitter can be a laser micro-ablation device, which emits a laser of a preset wavelength to remove stubborn deposits from the windshield.

[0053] Specifically, based on the type of obstruction and the pre-observed obstruction area of ​​the vehicle's windshield, the target cleaning device and its cleaning operation parameters can be determined. For example, when the obstruction type is rain or snow, the cleaning method can be determined to be a Level 1 response execution mode, and the target cleaning device can be identified as the windshield wipers, which will clear the rain and snow from the vehicle's windshield. Once the target cleaning device is identified as the windshield wipers, their cleaning operation parameters can also be determined, such as the target number of sprays, the target spray pattern, and the target oscillation pattern.

[0054] Then, based on the determined windshield wipers and their cleaning parameters, the wipers are controlled to spray water to clean the obstructions on the vehicle's windshield until they are completely removed, ensuring a clean windshield and a clear view for the driver.

[0055] In another alternative example, when the obstruction on the vehicle's windshield is large and the obstruction is attached to it, relying solely on windshield wipers is insufficient for quick removal. In this case, a three-level response execution mode can be determined, designating the laser emitter as the target cleaning device to remove rain and snow from the windshield. The cleaning parameters for the laser emitter can also be determined based on the obstruction area and the type of obstruction. For example, the cleaning parameters include the target spray path and target spray angle of the laser emitter.

[0056] When the obstruction area is small and the obstruction type is not rain or snow, the cleaning method can be determined as a level-two response, and the target cleaning device can be identified as a nitrogen injector. The cleaning operation parameters for the nitrogen injector can also be determined based on the type of obstruction. For example, the cleaning operation parameters include the target injection pressure, target injection time, and target injection mode of the nitrogen injector.

[0057] Then, based on the predetermined laser emitting device and its cleaning parameters, the laser emitting device is controlled to spray and clean the obstructions on the vehicle's windshield according to these parameters until the obstructions are completely removed, ensuring a clean windshield and clear visibility for the driver. Alternatively, based on the predetermined nitrogen injector and its cleaning parameters, the nitrogen injector is controlled to spray and clean the obstructions on the vehicle's windshield according to its parameters until the obstructions are completely removed, ensuring a clean windshield and clear visibility for the driver.

[0058] It is understood that the technical solution provided in this embodiment identifies whether there is an obstruction on the vehicle's windshield based on the acquired multimodal information of the vehicle's windshield, including windshield pressure information, radar point cloud information, and / or windshield images, thus obtaining an obstruction identification result. Furthermore, when the obstruction identification result indicates the presence of an obstruction, the type of obstruction is identified to obtain the obstruction category. Because the obstruction identification result is determined based on windshield pressure information, radar point cloud information, and / or windshield images, the obstruction identification result is more accurate than when relying solely on a single sensor for obstruction identification. Furthermore, when obstructions are present, the obtained obstruction category is more accurate. Moreover, by determining the target cleaning device and its cleaning operation parameters based on the obstruction category and the obstruction area of ​​the vehicle's windshield, the determined cleaning operation parameters of the target cleaning device can be matched with the obstruction category and obstruction area. By controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters, the obstruction of the vehicle's windshield can be quickly removed, ensuring the safe driving of the vehicle. This effectively solves the problem that the existing technology has a single method for cleaning obstructions on the vehicle's windshield and cannot quickly remove obstructions.

[0059] In one possible implementation, in step S11, based on the acquired multimodal information of the vehicle's windshield, the presence of an obstruction is identified to obtain an obstruction identification result; and when the obstruction identification result indicates the presence of an obstruction, the type of the obstruction is identified to obtain an obstruction category, including: Based on multimodal information of the vehicle's windshield, an initial identification of whether an obstruction exists on the windshield is performed, resulting in an initial obstruction identification result. This initial obstruction identification result includes a first identification result, a second identification result, and / or a third identification result. The first identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on windshield pressure information. The second identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on windshield images. The third identification result is obtained by identifying whether an obstruction exists on the vehicle's windshield based on radar point cloud information. Based on the initial obstruction identification result, the vehicle's windshield... The system comprehensively judges whether there is an obstruction on the window to obtain an obstruction identification result. When the obstruction identification result indicates the presence of an obstruction, it comprehensively judges the type of the obstruction based on the initial obstruction category to obtain the obstruction category. The initial obstruction category includes an obstruction first category, an obstruction second category, and / or an obstruction third category. The obstruction first category is obtained by identifying the type of obstruction based on the front window pressure information. The obstruction second category is obtained by identifying the type of obstruction based on the front window image. The obstruction third category is obtained by identifying the type of obstruction based on radar point cloud information.

[0060] Specifically, windshield pressure information can include the average pressure of multiple areas of the vehicle's windshield. Based on the average pressure of each area, the presence of obstructions can be determined. When the average pressure of an area differs significantly from that of other areas, an obstruction is confirmed to exist in that area. If an obstruction is present, its category can be determined based on the average pressure, resulting in an obstruction category I. For example, if the local pressure suddenly increases by 300 Pa within a short period, the obstruction category I is determined to be neither rain nor snow. When the pressure in an area increases uniformly throughout, the obstruction category I is determined to be rain or snow. When the average pressure of an area does not differ significantly from that of other areas, an obstruction is confirmed to exist in that area.

[0061] Based on the obstruction identification results for all areas, an initial identification is performed to determine whether an obstruction exists on the vehicle's windshield, yielding a first identification result. This first identification result includes whether an obstruction exists or not. If the obstruction identification result for any area of ​​the vehicle's windshield indicates the presence of an obstruction, the first identification result is determined to be "obstruction exists." If the obstruction identification result for all areas of the vehicle's windshield indicates the absence of an obstruction, the first identification result is determined to be "obstruction does not exist."

[0062] Based on the windshield image, deep learning and image recognition technologies can be used to perform a second identification to determine if there are obstructions on the vehicle's windshield, resulting in a second identification result. This second identification result includes whether an obstruction is present or not. When the second identification result indicates the presence of an obstruction, a second category of the obstruction can be determined directly from the windshield image. For example, the second category of the obstruction might be identified as a lightweight plastic bag, insect residue, or hail fragments.

[0063] Based on radar point cloud information, obstacles in front of the vehicle can be monitored, and the obstruction detection result of the vehicle's windshield can be predicted. This allows for a third identification of whether an obstruction exists on the vehicle's windshield, yielding a third identification result. This third identification result includes whether an obstruction exists or not. When it is predicted that an obstruction exists on the vehicle's windshield, the third identification result is "Obstruction exists." Alternatively, based directly on radar point cloud information, while determining the presence of an obstruction, a third category of the obstruction can be identified. When it is predicted that no obstruction exists on the vehicle's windshield, the third identification result is "No obstruction exists."

[0064] In an optional example, when monitoring obstacles in front of the vehicle based on radar point cloud information, if the probability of an obstacle colliding with the windshield is greater than a preset probability threshold, the third identification result is determined to be the presence of an obstruction. For example, the preset probability threshold could be 80%.

[0065] The first, second, and / or third identification results are used as the initial identification results for the obstruction. A comprehensive judgment is made based on these initial identification results to determine whether an obstruction exists on the vehicle's windshield, resulting in an obstruction identification result. When making a comprehensive judgment based on the first, second, and third identification results, the probability of an obstruction's presence and the probability of its absence can be statistically calculated. If the probability of an obstruction's presence is greater than or equal to the probability of its absence, the obstruction identification result is determined to be the presence of an obstruction. Alternatively, the obstruction identification result can be determined based on any two of the first, second, and third identification results, or any one of the first, second, and third identification results.

[0066] When the obstruction identification result indicates the presence of an obstruction, the first, second, and / or third categories of the obstruction are used as the initial category. The type of the obstruction is then comprehensively determined based on these initial categories to arrive at the obstruction category. For example, if the first category of the obstruction is rain or snow, the obstruction category can be determined as rain or snow. If both the second and third categories of the obstruction are lightweight plastic bags, the obstruction category is determined as lightweight plastic bags.

[0067] It should be noted that when determining the first, second, or third category of the occlusion, the probability of the first, second, or third category of the occlusion can also be obtained. Therefore, based on the probabilities of the first, second, and third categories of the occlusion, the category with the highest probability can be selected as the occlusion category.

[0068] Understandably, comprehensively judging the obstruction identification result by combining front window pressure information, radar point cloud information, and front window image can increase the accuracy of identification and avoid false triggering of target clearing devices. Compared with relying solely on front window image to judge the obstruction identification result, the reliability is significantly improved, especially in severe weather conditions such as heavy rain / fog, where the reliability is improved by 47%. Furthermore, it can respond promptly, with the response time shortened to 0.3 seconds, achieving millisecond-level obstruction identification.

[0069] In one possible implementation, step S13 involves controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters. Afterwards, the above-mentioned method for cleaning obstructions from the vehicle's windshield further includes: The temperature difference of the vehicle's windshield surface is obtained after cleaning; the temperature difference of the windshield surface is used to characterize the temperature difference between the obscured area and the unobscured area of ​​the vehicle's windshield; the obscured area is determined based on the location information of the obscured object; based on the temperature difference of the windshield surface, it is determined whether the obscured object has been completely cleaned; when it is determined that the obscured object has not been completely cleaned, the target cleaning device is controlled to clean the obscured object again according to the cleaning operation parameters.

[0070] Specifically, when the obstruction area of ​​the vehicle's windshield is less than a first preset threshold, the target cleaning device can be identified as a nitrogen injector. After controlling the nitrogen injector to spray and clean the obstruction according to the cleaning operation parameters, it can also be determined whether the spray cleaning has completely removed the obstruction.

[0071] In one optional example, the temperature difference of the vehicle's windshield surface after cleaning is obtained. The temperature difference can be the temperature difference between the obstructed and unobstructed areas of the windshield surface. The obstructed area refers to the area of ​​the windshield blocked by an object, and the unobstructed area is the area of ​​the windshield excluding the obstructed area. The obstructed area can be determined based on pre-acquired location information of the obstruction. The location information of the obstruction can be determined based on at least one of windshield pressure information, radar point cloud information, and windshield image, or it can be determined based on the outline information of the obstruction.

[0072] The temperature difference on the glass surface can be the difference between the temperature of the covered area and the temperature of the uncovered area of ​​the windshield of a vehicle.

[0073] Because the temperature of the glass surface in the obstructed area differs from that in the unobstructed area, this temperature difference can be used to determine whether the obstruction has been completely removed during cleaning. A significant temperature difference indicates that the obstruction on the vehicle's windshield has not been completely removed and remains. If the obstruction is not completely removed, its outline can be marked and displayed, for example, through an AR-HUD (Augmented Reality-Head-Up Display). The temperatures of the obstructed and unobstructed areas of the glass surface can be obtained using a thermal imager.

[0074] In an optional example, if the temperature difference (ΔT) on the glass surface is greater than a preset temperature difference threshold, it is determined that the obstruction has not been completely removed. If the temperature difference on the glass surface is less than or equal to the preset temperature difference threshold, it is determined that the obstruction has been completely removed. The preset temperature difference threshold can be 5°C or 2°C, and this embodiment does not specifically limit it.

[0075] If obstructions remain on the vehicle's windshield, the process is incomplete and requires further cleaning. During this second cleaning, the target cleaning device can be controlled to clean the obstructions according to the specified parameters. For example, a nitrogen injector can be controlled to spray and clean the obstructions again until they are completely removed. Once the obstructions on the windshield are cleared, the HUD (Head-Up Display) can be restored to normal operation. The HUD utilizes optical reflection principles combined with advanced display technology to project key vehicle information (such as speed, navigation, fuel consumption, and driver assistance warnings) onto the windshield in front of the driver's line of sight, allowing the driver to access this information without looking down, ensuring a clear field of vision and enhancing safe driving.

[0076] In one possible implementation, step S12 involves determining the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield, including: When the obstruction type is rain or snow, the target cleaning device is determined to be a windshield wiper; when the obstruction type is not rain or snow and the obstruction area is less than a first preset threshold, the target cleaning device is determined to be a nitrogen injector; when the obstruction type is not rain or snow and the obstruction area is greater than or equal to the first preset threshold, the target cleaning device is determined to be a laser emitting device.

[0077] Specifically, the target cleaning device can be determined based on the type of obstruction, the area of ​​obstruction, or a combination of the type of obstruction and the area of ​​obstruction on the vehicle's windshield.

[0078] Once the type of obstruction is determined, if it is rain or snow, since rain and snow can be quickly cleared by windshield wipers, the windshield wipers are selected as the target cleaning device. The wipers are then controlled to clear the rain and snow according to the cleaning parameters. These parameters can include the number of wipes and the cleaning mode. The number of wipes and the cleaning mode can also be determined based on the area of ​​the obstruction.

[0079] When the type of obstruction is determined to be neither rain nor snow, the target cleaning device can be determined based on the obstruction area, and the cleaning operation parameters of the target cleaning device can also be determined based on the type of obstruction. For example, when the obstruction area is less than a first preset threshold, the obstruction area is small, and a nitrogen sprayer can be used to spray and clean the obstruction, thus the target cleaning device is determined to be a nitrogen sprayer. The first preset threshold can be 70%, but this embodiment can set different first preset thresholds according to specific scenarios, and this embodiment does not specifically limit this. When the obstruction area is greater than or equal to the first preset threshold, since the obstruction area is large, a laser emitting device can be used for more efficient cleaning, thus the target cleaning device is determined to be a laser emitting device. Because nitrogen sprayer cleaning is less costly, it can be used to spray and clean obstructions with smaller areas to save costs.

[0080] In one possible implementation, when the target cleaning device is determined to be a nitrogen injector, the cleaning operation parameters for determining the target cleaning device include: Based on the type of obstruction, determine the target injection pressure, target injection time, and target injection mode of the nitrogen injector.

[0081] Specifically, when a nitrogen injector is identified as the target cleaning device, the target injection pressure, target injection time, and target injection mode of the nitrogen injector can be determined based on the type of obstruction. The airflow of the nitrogen injector can also be determined. For example, the nitrogen injector can be designed to remove small obstructions with an airflow of 15 MPa within 0.2 seconds. Localized cleaning of obstructions is also possible, with the cleaning time set to 200-300 milliseconds. The nitrogen injector is controlled to spray and clean obstructions from the vehicle's windshield using the target injection pressure, target injection time, and target injection mode. The target injection pressure, target injection time, and target injection mode differ for different types of obstructions, as shown in Table 1. Table 1 illustrates the correspondence between obstruction types and target injection pressure, target injection time, and target injection mode.

[0082] Table 1 shows the correspondence between obstruction type and target injection pressure, target injection time, and target injection mode. It should be noted that there are other categories of obstructions, which are not specifically limited in this embodiment. The corresponding target injection pressure, target injection time and target injection mode can also be set in advance.

[0083] In one possible implementation, when the obstruction identification result indicates the presence of an obstruction, the above-mentioned method for clearing obstructions from the vehicle's windshield further includes: Based on the front window image, determine the location information of the obstruction; based on the location information, determine the target angle of the target cleaning device; control the target cleaning device to clean the obstruction according to the target angle.

[0084] Specifically, when the nitrogen injector is identified as the target cleaning device, feature extraction and obstacle recognition can be performed based on the front window image to determine the location information of the obstruction. Then, the centroid coordinates of the obstruction are determined based on its location information. Assuming the centroid coordinates of the obstruction are (x, y), the deflection angle can be calculated using a PID control algorithm (Proportional-Integral-Derivative Controller) based on these coordinates. This deflection angle is then used as the target angle, and the injector's deflection angle is dynamically adjusted in real time according to the target angle to control the target cleaning device and clean the obstruction according to the target angle. The cleaning can be any one or more of water spray cleaning, air spray cleaning, or laser spray cleaning.

[0085] Calculating the deflection angle When this happens, the calculation can be performed using the following formula: ; in, The focal length of the camera lens used to capture the front window image. Let y be the centroid coordinate of the occluded object at the current moment. The initial centroid coordinate of the occluder is determined.

[0086] In one possible implementation, the above-mentioned method for clearing obstructions from the vehicle's windshield also includes: When the obstruction area is greater than or equal to the second preset threshold and less than the first preset threshold, the vehicle's front window visible area image, rearview mirror image, and blind spot monitoring image are acquired; the front window visible area image, rearview mirror image, and blind spot monitoring image are aligned and stitched together to obtain a virtual front view image; the virtual front view image is projected onto the vehicle's dashboard area for display.

[0087] Specifically, when the obstruction area of ​​the vehicle's windshield is greater than or equal to a second preset threshold and less than a first preset threshold, due to the large obstruction area, while clearing the obstruction, it is also necessary to provide the driver with a virtual environment image of the vehicle's surroundings so that the driver can assess the surrounding environment and ensure safe driving. The second preset threshold can be 30%, but this embodiment does not impose a specific limitation.

[0088] The vehicle's rearview mirrors integrate side-view cameras. When the obstruction area of ​​the vehicle's front window is greater than or equal to a second preset threshold and less than a first preset threshold, the side-view cameras are activated to monitor the vehicle's blind spots and obtain blind spot monitoring images. Rearview mirror images can also be obtained from the vehicle's electronic rearview mirrors. The front window image includes both obstructed and unobstructed areas. After feature extraction and cropping, the visible area image of the front window can be obtained. The visible area image of the front window represents the remaining visible area of ​​the vehicle's front window.

[0089] The images of the vehicle's front window visible area, rearview mirror, and blind spot monitoring are subjected to feature matching to achieve pixel-level alignment. These aligned images are then stitched together to obtain a virtual front view image. This virtual front view image integrates image information from multiple cameras around the vehicle and is projected onto the vehicle's dashboard for display, allowing the driver to obtain timely information about the vehicle's surroundings and ensuring driving safety. Feature matching can be performed using SIFT (Scale-Invariant Feature Transform) or other matching methods; this embodiment does not impose a specific limitation on this method.

[0090] In an optional example, the PSNR (Peak Signal-to-Noise Ratio) of the stitched virtual front view image is ≥32dB. The PSNR value measures the quality of the virtual front view image, evaluating the similarity between the front window visible area image, rearview mirror image, and blind spot monitoring image and the stitched virtual front view image. A higher PSNR value indicates better quality and less distortion. When the PSNR value of the virtual front view image is ≥32dB, it ensures that the stitching quality meets human visual requirements. The virtual front view image can also be a 180° wide-angle virtual forward view that conforms to the human eye's field of vision, allowing the driver to see the area in front of the vehicle without turning their head, improving driving safety and convenience.

[0091] When projecting the virtual front view image onto the vehicle's dashboard area, projection parameters can be set to better match the human eye's viewing angle. The specific projection parameter settings are shown in Table 2 below. Table 2 shows the target projection parameters for the virtual front view image.

[0092] Table 2. Target projection parameters of virtual front view image In another optional example, when the obstruction area is greater than or equal to a second preset threshold, vehicle sensors and other equipment are easily obstructed, affecting the safety of autonomous driving. If the vehicle is in autonomous driving mode, the steering wheel can vibrate to prompt the driver to take over manually, and the vibration will stop after the driver takes over. Furthermore, the electronic stability system limits the vehicle speed to 80 km / h, reducing the probability of rear-end collisions or other traffic accidents. The conditions for the steering wheel to vibrate and stop vibrating can be determined based on the steering wheel tactile recognition results. The specific code for steering wheel tactile recognition is as follows: # Example of vibration mode encoding based on CAN protocol def generate_vibration(obstacle_loc): zones = { 'left': [1,0,0,0, 0.5Hz], 'center':[0,1,0,0, 2.0Hz], 'right': [0,0,1,0, 0.5Hz] } send_CAN_msg(ID=0x3E5, Data=zones[obstacle_loc]).

[0093] In one possible implementation, the above-mentioned method for clearing obstructions from the vehicle's windshield also includes: When the obstructed area is greater than or equal to the first preset threshold, an emergency avoidance signal is sent to vehicles behind.

[0094] Specifically, when the obstructed area is greater than or equal to a first preset threshold, it indicates that the obstruction area is too large. In this case, the V2X (Vehicle-to-Everything) emergency communication protocol can be simultaneously activated to send an emergency avoidance signal to vehicles behind, enabling them to take timely measures to avoid danger. The emergency avoidance signal can be a broadcast message, including the coordinates, length, and speed of the obstructed vehicle. 3D alerts can also be played through the headrest speakers to alert the drivers of the obstructed vehicle and those behind, preventing panic caused by a full vehicle alarm.

[0095] In one possible implementation, when the target cleaning device is determined to be a laser emitting device, the cleaning operation parameters of the laser emitting device can also be determined. These parameters may include target ablation time, target laser power density, target wavelength, target spray path, and target removal area. For example, the target ablation time is 400-500 ms, the laser emitting device sprays laser light for 400-500 ms of micro-ablation removal, the target laser power density is ≤1.5 J / cm² (compliant with IEC 60825-1 Class 1 safety standard), the target wavelength is 355 nm (suitable for ablation of organic materials), and the target removal area is the area excluding sensors (rain sensors, light sensors, etc.). The target spray path can be determined based on the location information of obstructions using a greedy algorithm. The target spray path can be set using the following code: MATLAB % Generate target spray path [X,Y] = meshgrid(1:0.1:10); Z = peaks(X,Y); contour_levels = linspace(min(Z(:)), max(Z(:)), 5); optimized_path = greedy_contour_tracing(Z, contour_levels).

[0096] In one possible implementation, the vehicle is equipped with an OTA (Over-The-Air Technology) self-learning module. This module can track the frequency of driver manual intervention and determine the driver's driving style based on this frequency. Consequently, it can determine the response time thresholds for triggering the target clearing device and projecting the virtual forward-looking image, thus more effectively assisting the driver in driving safely and reducing traffic accidents. For example, when the driver's driving style is that of a novice, the virtual forward-looking image projection can be triggered 0.5 seconds in advance.

[0097] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0098] Corresponding to the method described in the above embodiments, Figure 2This diagram illustrates the structure of a vehicle windshield obstruction removal device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown.

[0099] Reference Figure 2 The vehicle windshield obstruction removal device 3 includes: The obstruction recognition module 31 is used to identify whether there is an obstruction on the vehicle's front window based on the acquired multimodal information of the vehicle's front window, and obtain an obstruction recognition result; and when the obstruction recognition result indicates that there is an obstruction, to identify the type of the obstruction and obtain the obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information and / or front window image; The cleaning parameter determination module 32 is used to determine the target cleaning device and the cleaning operation parameters of the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield; wherein, the target cleaning device includes at least one of a windshield wiper, a nitrogen injector and a laser emitting device; The control module 33 is used to control the target cleaning device to clean up the obstructions according to the cleaning operation parameters.

[0100] In one possible implementation, the obstruction recognition module 31 is further configured to perform initial recognition of whether an obstruction exists on the vehicle's front window based on the vehicle's front window multimodal information, and obtain an initial obstruction recognition result; wherein the initial obstruction recognition result includes a first recognition result, a second recognition result, and / or a third recognition result; the first recognition result is obtained by recognizing whether an obstruction exists on the vehicle's front window based on front window pressure information; the second recognition result is obtained by recognizing whether an obstruction exists on the vehicle's front window based on a front window image; the third recognition result is obtained by recognizing whether an obstruction exists on the vehicle's front window based on radar point cloud information; and the third recognition result is obtained by recognizing whether an obstruction exists on the vehicle's front window based on the obstruction... The initial object identification result comprehensively judges whether there is an obstruction on the vehicle's windshield, and obtains the obstruction identification result. When the obstruction identification result indicates the presence of an obstruction, the type of obstruction is comprehensively judged based on the initial obstruction category, and the obstruction category is obtained. The initial obstruction category includes obstruction category one, obstruction category two, and / or obstruction category three. Obstruction category one is obtained by identifying the type of obstruction based on windshield pressure information; obstruction category two is obtained by identifying the type of obstruction based on windshield image; and obstruction category three is obtained by identifying the type of obstruction based on radar point cloud information.

[0101] In one possible implementation, the control target cleaning device cleans the obstruction according to the cleaning operation parameters. Then, the control module 33 is also used to acquire the temperature difference of the glass surface of the vehicle's windshield after cleaning. The glass surface temperature difference is used to characterize the temperature difference between the obstructed area and the unobstructed area of ​​the vehicle's windshield. The obstructed area is determined based on the location information of the obstruction. Based on the glass surface temperature difference, it is determined whether the obstruction has been completely cleaned. When it is determined that the obstruction has not been completely cleaned, the control target cleaning device is used to clean the obstruction again according to the cleaning operation parameters.

[0102] In one possible implementation, the cleaning parameter determination module 32 is further configured to determine the target cleaning device as a windshield wiper when the obstruction type is rain or snow; determine the target cleaning device as a nitrogen injector when the obstruction type is not rain or snow and the obstruction area is less than a first preset threshold; and determine the target cleaning device as a laser emitting device when the obstruction type is not rain or snow and the obstruction area is greater than or equal to the first preset threshold.

[0103] In one possible implementation, the cleaning parameter determination module 32 is further configured to determine the target injection pressure, target injection time, and target injection mode of the nitrogen injector according to the type of obstruction when the target cleaning device is determined to be a nitrogen injector.

[0104] In one possible implementation, when the obstruction identification result indicates the presence of an obstruction, the cleaning parameter determination module 32 is further configured to determine the position information of the obstruction based on the front window image; and determine the target angle of the target cleaning device based on the position information. The control module 33 is also used to control the target cleaning device to clean the obstruction according to the target angle.

[0105] In one possible implementation, the control module 33 is further configured to acquire the front window visible area image, rearview mirror image, and blind spot monitoring image of the vehicle when the obstruction area is greater than or equal to a second preset threshold and less than a first preset threshold; align and stitch the front window visible area image, rearview mirror image, and blind spot monitoring image to obtain a virtual front view image; and project the virtual front view image onto the vehicle's dashboard area for display.

[0106] In one possible implementation, the control module 33 is also used to send an emergency avoidance signal to vehicles behind when the obstruction area is greater than or equal to a first preset threshold.

[0107] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0109] This application also provides a device for clearing obstructions from a vehicle's windshield. The device includes at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above-described method embodiments.

[0110] This application also provides a vehicle, including a device for clearing obstructions from the vehicle's windshield; wherein the device for clearing obstructions from the vehicle's windshield is used to perform the steps in the above-described method embodiments.

[0111] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0112] This application provides a computer program product that, when run on a vehicle windshield obstruction removal device, enables the vehicle windshield obstruction removal device to perform the steps described in the above-described method embodiments.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0114] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for cleaning obstructions from the windshield of a vehicle, characterized in that, include: Based on the acquired multimodal information of the vehicle's front window, the presence of obstructions on the vehicle's front window is identified to obtain an obstruction identification result; and when the obstruction identification result indicates the presence of an obstruction, the type of the obstruction is identified to obtain an obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information, and / or front window image; Based on the type of obstruction and the obstruction area of ​​the vehicle's windshield, a target cleaning device and its cleaning operation parameters are determined; wherein, the target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device; Control the target cleaning device to clean the obstruction according to the cleaning operation parameters.

2. The method for cleaning obstructions from a vehicle's windshield as described in claim 1, characterized in that, The process involves identifying whether an obstruction exists on the vehicle's windshield based on the acquired multimodal information, obtaining an obstruction identification result; and, when the obstruction identification result indicates the presence of an obstruction, identifying the type of the obstruction to obtain an obstruction category, including: Based on the multimodal information of the vehicle's windshield, an initial identification of whether there is an obstruction on the vehicle's windshield is performed to obtain an initial identification result of the obstruction; wherein the initial identification result of the obstruction includes a first identification result, a second identification result, and / or a third identification result; the first identification result is obtained by identifying whether there is an obstruction on the vehicle's windshield based on the windshield pressure information; the second identification result is obtained by identifying whether there is an obstruction on the vehicle's windshield based on the windshield image; the third identification result is obtained by identifying whether there is an obstruction on the vehicle's windshield based on the radar point cloud information; Based on the initial obstruction identification result, a comprehensive judgment is made on whether there is an obstruction on the vehicle's windshield to obtain the obstruction identification result; When the obstruction identification result indicates the presence of an obstruction, the type of the obstruction is comprehensively judged based on the initial obstruction category to obtain the obstruction category; wherein, the initial obstruction category includes an obstruction first category, an obstruction second category, and / or an obstruction third category; the obstruction first category is obtained by identifying the type of the obstruction based on the front window pressure information; the obstruction second category is obtained by identifying the type of the obstruction based on the front window image; the obstruction third category is obtained by identifying the type of the obstruction based on the radar point cloud information.

3. The method for cleaning obstructions from a vehicle's windshield as described in claim 1, characterized in that, The method further includes controlling the target cleaning device to clean the obstruction according to the cleaning operation parameters, and then the method further includes: The temperature difference of the glass surface of the vehicle's windshield after cleaning is obtained; wherein, the temperature difference of the glass surface is used to characterize the temperature difference between the obstructed area and the unobstructed area of ​​the vehicle's windshield; the obstructed area is determined based on the location information of the obstruction. Based on the temperature difference on the glass surface, determine whether the obstruction has been completely removed; When it is determined that the obstruction has not been completely cleared, the target cleaning device is controlled to clean the obstruction again according to the cleaning operation parameters.

4. The method for removing obstructions from the windshield of a vehicle as described in claim 1, characterized in that, The step of determining the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield includes: When the type of obstruction is rain or snow, the target cleaning device is determined to be a windshield wiper. When the type of obstruction is not rain or snow, and the obstruction area is less than a first preset threshold, the target cleaning device is determined to be a nitrogen injector. When the type of obstruction is not rain or snow, and the obstruction area is greater than or equal to the first preset threshold, the target cleaning device is determined to be a laser emitting device.

5. The method for cleaning obstructions from a vehicle's windshield as described in claim 1, characterized in that, When the target cleaning device is determined to be a nitrogen injector, the cleaning operation parameters of the target cleaning device are determined, including: Based on the type of obstruction, the target injection pressure, target injection time, and target injection mode of the nitrogen injector are determined respectively.

6. The method for cleaning obstructions from a vehicle's windshield as described in claim 1, characterized in that, When the occlusion identification result indicates the presence of an occlusion, the method further includes: Based on the front window image, determine the position information of the obstruction; Based on the location information, the target angle of the target cleaning device is determined; Control the target cleaning device to clean the obstruction according to the target angle.

7. The method for removing obstructions from the windshield of a vehicle as described in claim 4, characterized in that, The method further includes: When the obstruction area is greater than or equal to the second preset threshold and less than the first preset threshold, the vehicle's front window visible area image, rearview mirror image and blind spot monitoring image are acquired. The front window visible area image, the rearview mirror image, and the blind spot monitoring image are aligned and stitched together to obtain a virtual front view image; The virtual front view image is projected onto the vehicle's dashboard area for display.

8. The method for cleaning obstructions from a vehicle's windshield as described in claim 4, characterized in that, The method further includes: When the obstruction area is greater than or equal to the first preset threshold, an emergency avoidance signal is sent to vehicles behind.

9. A device for clearing obstructions from a vehicle's windshield, characterized in that, include: An obstruction recognition module is used to identify whether there is an obstruction on the vehicle's front window based on the acquired multimodal information of the vehicle's front window, and obtain an obstruction recognition result; and when the obstruction recognition result indicates the presence of an obstruction, to identify the type of the obstruction and obtain an obstruction category; wherein, the multimodal information of the vehicle's front window includes front window pressure information, radar point cloud information and / or front window image; The cleaning parameter determination module is used to determine the target cleaning device and the cleaning operation parameters of the target cleaning device based on the type of obstruction and the obstruction area of ​​the vehicle's windshield; wherein the target cleaning device includes at least one of a windshield wiper, a nitrogen injector, and a laser emitting device; The control module is used to control the target cleaning device to clean the obstruction according to the cleaning operation parameters.

10. A vehicle, characterized in that, The device includes a windshield obstruction removal device; wherein the windshield obstruction removal device is used to perform the method described in any one of claims 1-8.

Images