Front windshield cleaning method, apparatus, storage medium, and device

Abstract

This invention discloses a method, device, storage medium, and apparatus for cleaning a windshield. The method includes: analyzing image information of the windshield to obtain foreign object information; when the foreign object information is caused by weather factors, determining a corresponding sensor based on the weather factors, and analyzing the foreign object information based on the sensor to obtain a first analysis result; when the foreign object information is caused by driving environment factors, determining a target viewing area based on the windshield cleaning device; when foreign object information is detected in the target viewing area, analyzing the foreign object information in the target viewing area to obtain a second analysis result; and cleaning the windshield based on the first or second analysis result. This invention can perform analysis using sensors or can determine and analyze the target viewing area where the foreign object information is located using cleaning equipment. Different analysis results allow for more targeted cleaning of dirt, improving the cleaning effect.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to a method, apparatus, storage medium, and device for cleaning a windshield. Background Technology

[0002] The windshield is crucial for driving safety. When dirt or grime accumulates on the windshield, the wipers cannot clean it thoroughly and effectively, obstructing the driver's view and posing a safety hazard.

[0003] Therefore, adjusting the windshield wipers is particularly important. Currently, it is impossible to perform cleaning work by adjusting the wipers in a targeted manner, resulting in incomplete cleaning. For example, it is impossible to accurately control the wipers because of obstructions.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, storage medium, and device for cleaning windshields, aiming to solve the technical problem of poor cleaning effect caused by the inability to effectively control the wipers for cleaning.

[0006] To achieve the above objectives, the present invention provides a method for cleaning a windshield, the method comprising the following steps:

[0007] The image information of the windshield is analyzed to obtain information about foreign objects in the image information;

[0008] When the foreign object information is caused by weather factors, the corresponding sensor is determined according to the weather factors, and the foreign object information is analyzed according to the sensor to obtain a first analysis result;

[0009] When the foreign object information is caused by driving environment factors, the target visual area is determined based on the windshield cleaning equipment;

[0010] When the foreign object information is detected in the target line of sight area, the foreign object information in the target line of sight area is analyzed to obtain a second analysis result;

[0011] The windshield is cleaned based on the first or second analysis results.

[0012] Optionally, before the step of analyzing the image information of the windshield to obtain foreign object information in the image information, the method further includes:

[0013] The convolutions of the residual network are determined based on the training set of weather images, and the objective function and optimizer are also determined.

[0014] The image recognition model to be processed is determined based on a preset image recognition model and according to the convolution, the objective function, and the optimizer.

[0015] The image recognition model to be processed is trained based on the training set;

[0016] When the training results reach the preset evaluation index, the target image recognition model is obtained;

[0017] Accordingly, the step of analyzing the image information of the windshield to obtain foreign object information in the image information includes:

[0018] The image information of the windshield is analyzed based on the target image recognition model to obtain information about foreign objects in the image information.

[0019] Optionally, the windshield cleaning method is applied to a windshield cleaning device, which includes an eye tracker and a position sensor.

[0020] When the foreign object information is caused by driving environment factors, the step of determining the target visibility area based on the windshield cleaning equipment includes:

[0021] When the foreign object information is caused by driving environment factors, the driver's driving observation angle is determined according to the eye tracker;

[0022] The driver's position information is determined based on the position sensor;

[0023] The driver's target line of sight on the windshield is determined based on the driving observation angle and the position information.

[0024] Optionally, the step of determining the driver's driving observation angle based on the eye tracker when the foreign object information is caused by driving environment factors includes:

[0025] When the driver's line of sight is detected to be on the windshield, the eye tracker is controlled to emit electromagnetic waves toward the driver's eyes;

[0026] The driver receives the reflected waves from the eyeball and determines the driving observation angle based on the light intensity of the reflected waves.

[0027] Optionally, the step of determining the driver's target line-of-sight area on the windshield based on the driving observation angle and the position information includes:

[0028] The driver's gaze duration on each area of ​​the windshield is calculated based on the driving observation angle, and the target gaze area is determined based on the gaze duration.

[0029] The driver's target line of sight on the windshield is determined based on the location information and the target gaze area.

[0030] Optionally, the step of analyzing the image information of the windshield to obtain foreign object information in the image information includes:

[0031] Compare the image information with the image information in the preset database;

[0032] Foreign object information in the image information is determined based on the comparison results.

[0033] Optionally, the step of cleaning the windshield based on the second analysis result includes:

[0034] The impact level of foreign object information within the target field of vision area on the driver is determined based on the second analysis results;

[0035] The windshield is cleaned by adjusting the cleaning function of the wipers according to the level of impact.

[0036] Furthermore, to achieve the above objectives, the present invention also proposes a windshield cleaning device, the windshield cleaning device including a memory, a processor, and a windshield cleaning program stored in the memory and capable of running on the processor, the windshield cleaning program being configured to implement the windshield cleaning method as described above.

[0037] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a windshield cleaning program, which, when executed by a processor, implements the windshield cleaning method as described above.

[0038] In addition, to achieve the above objectives, the present invention also proposes a windshield cleaning device, which includes: a foreign object information acquisition module, a foreign object information analysis module, a line-of-sight area determination module, and a cleaning module.

[0039] The foreign object information acquisition module is used to analyze the image information of the windshield and obtain the foreign object information in the image information;

[0040] The foreign object information analysis module is used to determine the corresponding sensor based on the weather factors when the foreign object information is caused by weather factors, and to analyze the foreign object information based on the sensor to obtain a first analysis result;

[0041] The line-of-sight area determination module is used to determine the target line-of-sight area based on the windshield cleaning equipment when the foreign object information is caused by driving environment factors.

[0042] The foreign object information analysis module is used to analyze the foreign object information in the target line of sight area to obtain a second analysis result when the foreign object information is detected in the target line of sight area.

[0043] The cleaning module is used to clean the windshield based on the first analysis result or the second analysis result.

[0044] This invention discloses a method, device, storage medium, and apparatus for cleaning a windshield. The method includes: analyzing image information of the windshield to obtain foreign object information in the image information; when the foreign object information is caused by weather factors, determining a corresponding sensor based on the weather factors, and analyzing the foreign object information based on the sensor to obtain a first analysis result; when the foreign object information is caused by driving environment factors, determining a target viewing area based on the windshield cleaning device; when foreign object information is detected in the target viewing area, analyzing the foreign object information in the target viewing area to obtain a second analysis result; and cleaning the windshield based on the first analysis result or the second analysis result. This invention employs different analysis methods to clean foreign objects caused by different factors. When the foreign object is caused by weather factors, the invention analyzes the foreign object information using sensors to obtain a first analysis result and then cleans the windshield based on that result. When the foreign object is caused by driving environment factors, the invention determines the driver's target visual area using a windshield cleaning device, analyzes the foreign objects within that area to obtain a second analysis result, and then cleans the windshield based on that result. This allows for targeted control of the wipers to clean dirt and improves the cleaning effect. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of the windshield cleaning device in the hardware operating environment of the embodiment of the present invention;

[0046] Figure 2 This is a schematic flowchart of the first embodiment of the windshield cleaning method of the present invention;

[0047] Figure 3 This is a flowchart illustrating the second embodiment of the windshield cleaning method of the present invention;

[0048] Figure 4 This is a flowchart illustrating the third embodiment of the windshield cleaning method of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the foreign object cleaning process of an embodiment of the windshield cleaning method of the present invention.

[0050] Figure 6This is a key observation area diagram showing the cleaning of foreign objects in an embodiment of the windshield cleaning method of the present invention;

[0051] Figure 7 This is a structural block diagram of the first embodiment of the windshield cleaning device of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] Reference Figure 1 , Figure 1 This is a schematic diagram of the windshield cleaning device structure in the hardware operating environment of the embodiment of the present invention.

[0055] like Figure 1 As shown, the windshield cleaning device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the windshield cleaning device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a windshield cleaning program.

[0058] exist Figure 1In the windshield cleaning device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user device; the windshield cleaning device calls the windshield cleaning program stored in the memory 1005 through the processor 1001 and executes the windshield cleaning method provided in the embodiment of the present invention.

[0059] Based on the above hardware structure, an embodiment of the windshield cleaning method of the present invention is proposed.

[0060] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the windshield cleaning method of the present invention, which presents the first embodiment of the windshield cleaning method of the present invention.

[0061] Step S10: Analyze the image information of the windshield to obtain information about foreign objects in the image information.

[0062] It should be noted that the execution subject of this embodiment may be a computer software service device with data processing, network communication and program running functions, such as a windshield cleaner, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0063] It should be understood that, because it is impossible to control the wipers for cleaning in a targeted manner, that is, it is impossible to perform targeted local or comprehensive cleaning based on the driver's needs and the surrounding environmental factors, and it is also impossible to determine the key areas to be cleaned.

[0064] To overcome the above-mentioned defects, this embodiment acquires image information of the windshield using a camera device, analyzes the image information to obtain foreign object information, and when the presence of foreign objects on the windshield is caused by weather factors, a corresponding sensor is determined according to different weather conditions. The sensor analyzes the foreign object information to obtain a first analysis result, and the windshield is cleaned according to the first analysis result. When the presence of foreign objects on the windshield is caused by driving environment factors, the windshield cleaning device determines the driver's key visual area. When foreign object information is detected in the key visual area, the foreign object information in the key visual area is analyzed to obtain a second analysis result, and the windshield is cleaned according to the second analysis result.

[0065] It is understandable that the image information of the windshield can be obtained by capturing images through cameras, either mounted on the windshield or mounted on the edge of the windshield.

[0066] It should be noted that image information can be analyzed based on image recognition models, or the image information can be compared with image information in a database to obtain information about foreign objects.

[0067] It is understood that foreign object information can be dirt or grime on the windshield, or rainwater, snowflakes, etc. This embodiment does not impose any limitations.

[0068] Furthermore, in order to accurately identify foreign objects on the windshield for targeted cleaning, step S10 of this embodiment may include:

[0069] Compare the image information with the image information in the preset database;

[0070] Foreign object information in the image information is determined based on the comparison results.

[0071] It should be noted that the camera mounted on the edge of the windshield will periodically take pictures of the windshield to collect information on the current cleanliness of the glass. The image data processing module converts the images into data, and compares the converted data with the data on the location of dirt and foreign objects stored in the database. Based on the comparison results, information such as whether there is dirt or foreign objects on the glass, the location of the dirt or foreign objects, and the area of ​​the windshield that is obstructed can be obtained.

[0072] Step S20: When the foreign object information is caused by weather factors, determine the corresponding sensor based on the weather factors, and analyze the foreign object information based on the sensor to obtain a first analysis result.

[0073] It should be noted that the image information on the windshield can be judged based on the image recognition model. If the image information on the windshield is determined to be rain, snow, etc., it indicates that the foreign object is caused by the weather. The foreign object information is analyzed by the sensors corresponding to different weather conditions to obtain the first analysis result, and cleaning is carried out based on the first analysis result.

[0074] It should be noted that when the system determines that the debris on the windshield is rainwater, it dictates that the weather is rainy. The rain sensor analyzes the rainfall intensity and activates the wipers at different frequencies depending on the amount of rain. Conversely, if the debris is determined to be foggy or snowy, the system compares the temperature and humidity inside and outside the vehicle using temperature and humidity sensors. When there is a significant difference in temperature and humidity, the air conditioning will activate and blow hot air onto the windshield while cleaning, maintaining a stable temperature and preventing fogging that could obstruct the driver's view.

[0075] Step S30: When the foreign object information is caused by driving environment factors, determine the target visual area according to the windshield cleaning equipment.

[0076] Understandably, when the image recognition model fails to identify the image information of the windshield, it compares the image information with the image information in the preset database. This can determine the type of foreign object, which could be debris attached to the windshield during driving, as well as the location and area of ​​obstruction.

[0077] It should be noted that the windshield cleaning device can be an eye tracker and a position sensor. The eye tracker, mounted above the dashboard, tracks the driver's eye gaze angle, and the position sensor collects the position information of the driver's seat (such as the distance between the driver and the passenger, seat height, and seat back angle). The driver's driving posture and field of vision are calculated based on the above information data, and the key observation area of ​​the windshield is determined according to the driver's field of vision.

[0078] It should be noted that the target visual area is the area that the driver focuses on observing while driving. This is achieved by sending electromagnetic waves to the driver's eyes and receiving the reflected waves through an eye tracker. The driver's gaze area is determined based on the emitted waves, and the driver's key observation area is determined based on the driver's seating position.

[0079] Step S40: When the foreign object information is detected in the target line of sight area, the foreign object information in the target line of sight area is analyzed to obtain a second analysis result.

[0080] It should be noted that after detecting foreign object information within the target line of sight area, the system analyzes the location of the foreign object within the target line of sight area and the area it obstructs to determine the degree of impact of the foreign object on the driver's vision. Based on different degrees of impact, the system can control the windshield wipers to operate in a targeted manner.

[0081] It is understandable that different priorities are determined based on the target's position within the line of sight and the degree of obstruction, with different priorities reflecting different degrees of impact on the driver's vision.

[0082] Furthermore, in order to improve the cleaning effect, step S40 of this embodiment may include:

[0083] The impact level of foreign object information within the target field of vision area on the driver is determined based on the second analysis results;

[0084] The windshield is cleaned by adjusting the cleaning function of the wipers according to the level of impact.

[0085] Understandably, if the foreign object is located in the exact center of the target's line of sight and obstructs a large area, it should be cleaned as the first priority, as it has the greatest impact on the driver's vision. If the foreign object is located in the exact center of the target's line of sight but obstructs a small area, it should be cleaned as the second priority. If the foreign object is located at the edge of the target's line of sight but obstructs a large area, it should be cleaned as the third priority. If the foreign object is located at the edge of the target's line of sight but obstructs a small area, it should be cleaned as the fourth priority.

[0086] Understandably, the first priority level has the greatest impact on the driver's visibility, while the second, third, and fourth priorities have a decreasing impact on the driver's visibility in that order.

[0087] It's important to note that the wiper's working range is determined based on the degree of obstruction. The wiper's oscillation angle and frequency are controlled to focus on cleaning areas of the windshield that severely impair the driver's vision. Specifically, this is achieved by adjusting the wiper's oscillation angle, frequency, and duration. The larger the area obstructed by dirt or debris, the larger the wiper's working range and the higher its frequency. Once the dirt or debris is removed, the cleaning process automatically stops. If the dirt or debris cannot be removed within 3 minutes, an alarm will be triggered on the driver.

[0088] Step S50: Clean the windshield according to the first analysis result or the second analysis result.

[0089] In specific implementations, for example, when the target image recognition model determines that the foreign object is snow, the first analysis result can be based on comparing the ambient temperature inside and outside the vehicle with that of the temperature and humidity sensor. When the temperature and humidity inside the vehicle differ significantly, the air conditioner is turned on to blow hot air onto the windshield to prevent it from fogging up. When the comparison with the database determines that the foreign object is caused by environmental factors during driving, the windshield is cleaned by controlling the swing angle, swing frequency, and swing time of the wipers according to the cleaning priority.

[0090] This embodiment analyzes image information from the windshield to obtain foreign object information. When the foreign object information is caused by weather factors, a corresponding sensor is determined based on the weather factors, and the foreign object information is analyzed based on the sensor to obtain a first analysis result. When the foreign object information is caused by driving environment factors, a target viewing area is determined based on the windshield cleaning device. When foreign object information is detected in the target viewing area, the foreign object information in the target viewing area is analyzed to obtain a second analysis result. The windshield is cleaned based on either the first or second analysis result. This embodiment uses different analysis methods for cleaning based on different factors causing foreign objects. When the foreign object is caused by weather factors, only the first analysis result is obtained from the sensor analysis of the foreign object information, and cleaning is performed based on the first analysis result. When the foreign object is caused by driving environment factors, the target viewing area of ​​the driver is determined based on the windshield cleaning device, the foreign object in the target viewing area is analyzed to obtain a second analysis result, and the windshield is cleaned based on the second analysis result. This allows for targeted control of the wipers to clean dirt, improving the cleaning effect.

[0091] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the windshield cleaning method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the windshield cleaning method of the present invention.

[0092] In the second embodiment, before step S10, the method further includes:

[0093] Step S00: Determine the convolution of the residual network based on the training set of weather images, and determine the objective function and optimizer.

[0094] Understandably, the training set of weather images includes weather image data such as sunny days, rainy days, cloudy days, snowy days, and dust storms, and the training set of weather images undergoes data cleaning and visualization analysis.

[0095] It should be noted that the images are pre-cropped to a uniform size of 3*224*224, and the data is split to create validation and training sets. In the residual structure network, a 1*1 convolutional kernel is first used for convolution, followed by M 3*3 convolutions, where M is determined by the number of images in the input dataset. Each of the M convolutional kernels is responsible for only a part of the data, and finally the extracted features are concatenated.

[0096] It should be noted that the objective function can be a loss function, a training function used to record loss and accuracy, or a validation function.

[0097] Step S01: Determine the image recognition model to be processed based on the preset image recognition model and according to the convolution, the objective function, and the optimizer.

[0098] It's important to note that some settings need to be configured before training the model. The loss function measures the model's accuracy during training. The optimizer determines how the model updates based on the data it sees and its own loss function. Metrics are used to monitor the training and testing phases. Prepare the loss function and optimizer, write the training and validation functions, and record the loss and accuracy.

[0099] Step S02: Train the image recognition model to be processed based on the training set.

[0100] Understandably, after training the image recognition model on a large training set, the model achieves high image recognition accuracy. The weights and other parameters obtained during training are then applied to classifying driving environment weather images. This allows vehicle cameras to collect weather image information, which can then be quickly identified using the model, achieving good accuracy and low error, and effectively classifying the various weather conditions trained on.

[0101] Step S03: When the training results reach the preset evaluation index, the target image recognition model is obtained.

[0102] It should be noted that during the iterative training of the image recognition model to be processed, adjustments are made continuously during training. The accuracy of the image recognition model to be processed is continuously saved according to the verification function. Appropriate evaluation metrics are set to evaluate the accuracy. When the accuracy of the model training reaches the preset evaluation metrics, training is stopped, and the target image recognition model is obtained.

[0103] Accordingly, the step of analyzing the image information of the windshield to obtain foreign object information in the image information includes:

[0104] The image information of the windshield is analyzed based on the target image recognition model to obtain information about foreign objects in the image information.

[0105] Understandably, a well-trained image recognition model possesses a certain level of image recognition accuracy. Therefore, it can effectively analyze the image information of the windshield and determine the current weather conditions based on the windshield image information. For example, if the image analysis results indicate that the foreign object is a raindrop, then the current weather is determined to be rainy.

[0106] This embodiment determines the convolution of the residual network based on the training set of weather images, and also determines the objective function and optimizer. Based on a preset image recognition model and according to the convolution, objective function, and optimizer, it determines the image recognition model to be processed. The image recognition model to be processed is trained using the training set. When the training result reaches a preset evaluation index, the target image recognition model is obtained. This embodiment determines the target image recognition model based on the residual network, objective function, optimizer, and preset image recognition model, and determines the accuracy of model training based on the objective function. When the model training accuracy reaches a preset evaluation index, model training is complete. Analyzing image information using the trained image recognition model can more accurately identify foreign objects in the image information, improving the accuracy of image recognition.

[0107] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the windshield cleaning method of the present invention, based on the above. Figure 2 The first embodiment shown presents a third embodiment of the windshield cleaning method of the present invention.

[0108] In the third embodiment, step S30 includes:

[0109] Step S301: When the foreign object information is caused by driving environment factors, determine the driver's driving observation angle according to the eye tracker.

[0110] Understandably, by taking images of the windshield using a camera mounted on the edge of the windshield, and analyzing whether the foreign object information was caused by driving environment factors, the eye tracker and position sensor are used to determine the key observation areas of the driver during driving.

[0111] Understandably, drivers need to constantly observe their surroundings while driving. Therefore, some areas are key areas for drivers to observe. By determining the driver's observation angle using an eye tracker, the key areas for observation can be identified based on the observation angle and the driver's position information. Foreign objects appearing in these key areas can then be cleaned more thoroughly.

[0112] It should be noted that the driving observation angle is the angle of the driver's maximum observation range during driving. The driver's observation area is determined based on the driving observation angle, and the driver's key observation area can be determined based on the duration of the observation area.

[0113] Furthermore, in order to meet different cleaning needs and save resources, step S301 of this embodiment may include:

[0114] When the driver's line of sight is detected to be on the windshield, the eye tracker is controlled to emit electromagnetic waves toward the driver's eyes;

[0115] The driver receives the reflected waves from the eyeball and determines the driving observation angle based on the light intensity of the reflected waves.

[0116] It should be noted that the eye tracker projects electromagnetic waves onto the eyeball and receives the reflected waves that are projected onto the eyeball. The gaze angle is determined based on the light intensity of the reflected waves received by the eye tracker when the eyeball is focused on the area of ​​the windshield.

[0117] Step S302: Determine the driver's position information based on the position sensor.

[0118] It is understandable that the driver's seat position information is determined by obtaining information such as seat height and seat back from position sensors.

[0119] Step S303: Determine the target line of sight area of ​​the driver on the windshield based on the driving observation angle and the position information.

[0120] Understandably, the area a driver observes varies depending on their position; for example, the further forward the seat is, the smaller the observed area, while the further back the seat is, the larger the observed area. Therefore, it is necessary to combine the driving observation angle and the driver's position information to determine the key observation area on the windshield for the driver.

[0121] Furthermore, in order to meet different cleaning needs and save resources, step S303 of this embodiment may include:

[0122] The driver's gaze duration on each area of ​​the windshield is calculated based on the driving observation angle, and the target gaze area is determined based on the gaze duration.

[0123] The driver's target line of sight on the windshield is determined based on the location information and the target gaze area.

[0124] It should be noted that the driving observation angle is the maximum range of the driver's line of sight. Within the driving observation angle, the driver's gaze duration in each area is recorded. Different gaze durations indicate that if the driver gazes for a long time, it means that the driver will focus on that area. If there are foreign objects in that area, targeted cleaning is required.

[0125] For ease of understanding, please refer to Figure 5 and Figure 6 To explain, Figure 5 This is a diagram illustrating the process of cleaning foreign objects. Figure 6 This is a diagram showing the cleaning of foreign objects in the key observation area. Figure 5The windshield wiper cleans foreign objects within its working range. After identifying the key observation area, the working range of the wiper is adjusted, no longer cleaning a large area, but only cleaning the area where foreign objects appear. After cleaning the key observation area, the wiper cleans the remaining areas where foreign objects appear.

[0126] In this embodiment, when the foreign object information is caused by driving environment factors, the driver's driving observation angle is determined based on the eye tracker; the driver's position information is determined based on the position sensor; and the driver's target visual area on the windshield is determined based on the driving observation angle and the position information. This embodiment determines the driver's driving observation angle based on the eye tracker, determines the driver's position information based on the position sensor, and combines the driving observation angle and position information to determine the driver's key observation area on the windshield. This allows the wipers to be controlled to clean more effectively based on the key observation area, improving cleaning efficiency.

[0127] Furthermore, embodiments of the present invention also propose a storage medium storing a windshield cleaning program, which, when executed by a processor, implements the windshield cleaning method as described above.

[0128] In addition, refer to Figure 7 The present invention also proposes a windshield cleaning device, which includes: a foreign object information acquisition module 10, a foreign object information analysis module 20, a line-of-sight area determination module 30, and a cleaning module 40.

[0129] The foreign object information acquisition module 10 is used to analyze the image information of the windshield and obtain the foreign object information in the image information;

[0130] The foreign object information analysis module 20 is used to determine the corresponding sensor based on the weather factors when the foreign object information is caused by weather factors, and to analyze the foreign object information based on the sensor to obtain a first analysis result.

[0131] The line-of-sight area determination module 30 is used to determine the target line-of-sight area based on the windshield cleaning device when the foreign object information is caused by driving environment factors.

[0132] The foreign object information analysis module 20 is used to analyze the foreign object information in the target line of sight area to obtain a second analysis result when the foreign object information is detected in the target line of sight area;

[0133] The cleaning module 40 is used to clean the windshield based on the first analysis result or the second analysis result.

[0134] This invention discloses a method, device, storage medium, and apparatus for cleaning a windshield. The method includes: analyzing image information of the windshield to obtain foreign object information in the image information; when the foreign object information is caused by weather factors, determining a corresponding sensor based on the weather factors, and analyzing the foreign object information based on the sensor to obtain a first analysis result; when the foreign object information is caused by driving environment factors, determining a target viewing area based on the windshield cleaning device; when foreign object information is detected in the target viewing area, analyzing the foreign object information in the target viewing area to obtain a second analysis result; and cleaning the windshield based on the first analysis result or the second analysis result. This invention employs different analysis methods to clean foreign objects caused by different factors. When the foreign object is caused by weather factors, the invention analyzes the foreign object information using sensors to obtain a first analysis result and then cleans the windshield based on that result. When the foreign object is caused by driving environment factors, the invention determines the driver's target visual area using a windshield cleaning device, analyzes the foreign objects within that area to obtain a second analysis result, and then cleans the windshield based on that result. This allows for targeted control of the wipers to clean dirt and improves the cleaning effect.

[0135] Based on the first embodiment of the windshield cleaning device of the present invention described above, a second embodiment of the windshield cleaning device of the present invention is proposed.

[0136] In this embodiment, the windshield cleaning device further includes a model determination module 00, which is used to determine the convolution of the residual network based on the training set of weather images, and to determine the objective function and optimizer.

[0137] Furthermore, the model determination module 00 is also used to determine the image recognition model to be processed based on a preset image recognition model and according to the convolution, the objective function, and the optimizer.

[0138] Furthermore, the model determination module 00 is also used to train the image recognition model to be processed based on the training set.

[0139] Furthermore, the model determination module 00 is also used to obtain a target image recognition model when the training result reaches a preset evaluation value index.

[0140] Furthermore, the model determination module 00 is also used to obtain a target image recognition model when the training result reaches a preset evaluation value index.

[0141] Furthermore, the line-of-sight area determination module 30 is also used to determine the driver's driving observation angle based on the eye tracker when the foreign object information is caused by driving environment factors.

[0142] Furthermore, the line-of-sight area determination module 30 is also used to determine the position information of the driver's seat based on the position sensor.

[0143] Furthermore, the line-of-sight area determination module 30 is also used to determine the target line-of-sight area of ​​the driver on the windshield based on the driving observation angle and the position information.

[0144] Furthermore, the line-of-sight area determination module 30 is also used to control the eye tracker to emit electromagnetic waves toward the driver's eyes when the driver's line of sight is detected to be on the windshield.

[0145] Furthermore, the line-of-sight area determination module 30 is also used to receive the reflected waves from the eyeball and determine the driver's driving observation angle based on the light intensity of the reflected waves.

[0146] Furthermore, the line-of-sight area determination module 30 is also used to calculate the duration of the driver's gaze on each area on the windshield based on the driving observation angle, and to determine the target gaze area based on the gaze duration.

[0147] Furthermore, the line-of-sight area determination module 30 is also used to determine the driver's target line-of-sight area on the windshield based on the position information and the target gaze area.

[0148] Furthermore, the foreign object information acquisition module 10 is also used to compare the image information with the image information in a preset database.

[0149] Furthermore, the foreign object information acquisition module 10 is also used to determine the foreign object information in the image information based on the comparison results.

[0150] Furthermore, the cleaning module 40 is also used to determine the impact level of foreign object information in the target field of vision area on the driver based on the second analysis result.

[0151] Furthermore, the cleaning module 40 is also used to adjust the cleaning function of the windshield wipers to clean the windshield according to the impact level.

[0152] Other embodiments or specific implementations of the windshield cleaning device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0154] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0156] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for cleaning a windshield, characterized in that, The windshield cleaning method includes the following steps: The image information of the windshield is analyzed to obtain information about foreign objects in the image information; When the foreign object information is caused by weather factors, the corresponding sensor is determined according to the weather factors, and the foreign object information is analyzed according to the sensor to obtain a first analysis result; When the foreign object information is caused by driving environment factors, the target visual area is determined based on the windshield cleaning equipment; When the foreign object information is detected in the target line of sight area, the foreign object information in the target line of sight area is analyzed to obtain a second analysis result; Clean the windshield according to the first analysis result or the second analysis result; The windshield cleaning method is applied to a windshield cleaning device, which includes a position sensor. When the foreign object information is caused by driving environment factors, the step of determining the target visibility area based on the windshield cleaning equipment includes: When the foreign object information is caused by driving environment factors, determine the driver's driving observation angle; The driver's position information is determined based on the position sensor; The driver's target line of sight on the windshield is determined based on the driving observation angle and the position information. The step of determining the driver's target line-of-sight area on the windshield based on the driving observation angle and the position information includes: The driver's gaze duration on each area of ​​the windshield is calculated based on the driving observation angle, and the target gaze area is determined based on the gaze duration. The driver's target line of sight on the windshield is determined based on the location information and the target gaze area; The step of cleaning the windshield based on the second analysis result includes: The degree of impact of foreign object information within the target visual area on the driver's vision is determined based on the second analysis result, whereby the second analysis result is the location of the foreign object within the target visual area and the area it obstructs. Adjust the oscillation angle, oscillation frequency, and oscillation time of the windshield wipers when cleaning the windshield according to the degree of obstruction to the view.

2. The windshield cleaning method as described in claim 1, characterized in that, Before the step of analyzing the image information of the windshield to obtain the foreign object information in the image information, the method further includes: The convolutions of the residual network are determined based on the training set of weather images, and the objective function and optimizer are also determined. The image recognition model to be processed is determined based on a preset image recognition model and according to the convolution, the objective function, and the optimizer. The image recognition model to be processed is trained based on the training set; When the training results reach the preset evaluation index, the target image recognition model is obtained; Accordingly, the step of analyzing the image information of the windshield to obtain foreign object information in the image information includes: The image information of the windshield is analyzed based on the target image recognition model to obtain information about foreign objects in the image information.

3. The windshield cleaning method as described in claim 1, characterized in that, The windshield cleaning device also includes: an eye tracker; The step of determining the driver's observation angle when the foreign object information is caused by driving environment factors includes: When the foreign object information is caused by driving environment factors, the driver's driving observation angle is determined based on the eye tracker.

4. The windshield cleaning method as described in claim 3, characterized in that, When the foreign object information is caused by driving environment factors, the step of determining the driver's driving observation angle based on the eye tracker includes: When the driver's line of sight is detected to be on the windshield, the eye tracker is controlled to emit electromagnetic waves toward the driver's eyes; The driver receives the reflected waves from the eyeball and determines the driving observation angle based on the light intensity of the reflected waves.

5. The windshield cleaning method according to any one of claims 1 to 4, characterized in that, The step of analyzing the image information of the windshield to obtain information about foreign objects in the image information includes: Compare the image information with the image information in the preset database; Foreign object information in the image information is determined based on the comparison results.

6. A windshield cleaning device, characterized in that, The windshield cleaning device includes: a memory, a processor, and a windshield cleaning program stored in the memory and executable on the processor, wherein the windshield cleaning program, when executed by the processor, implements the steps of the windshield cleaning method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium stores a windshield cleaning program, which, when executed by a processor, implements the steps of the windshield cleaning method as described in any one of claims 1 to 5.

8. A windshield cleaning device, characterized in that, The windshield cleaning device includes: a foreign object information acquisition module, a foreign object information analysis module, a line-of-sight area determination module, and a cleaning module; The foreign object information acquisition module is used to analyze the image information of the windshield and obtain the foreign object information in the image information; The foreign object information analysis module is used to determine the corresponding sensor based on the weather factors when the foreign object information is caused by weather factors, and to analyze the foreign object information based on the sensor to obtain a first analysis result; The line-of-sight area determination module is used to determine the target line-of-sight area based on the windshield cleaning equipment when the foreign object information is caused by driving environment factors. The foreign object information analysis module is used to analyze the foreign object information in the target line of sight area to obtain a second analysis result when the foreign object information is detected in the target line of sight area. The cleaning module is used to clean the windshield based on the first analysis result or the second analysis result; The windshield cleaning device is applied to a windshield cleaning equipment, which includes: a position sensor; The line-of-sight area determination module is also used to determine the driver's driving observation angle when the foreign object information is caused by driving environment factors; determine the driver's position information based on the position sensor; and determine the driver's target line-of-sight area on the windshield based on the driving observation angle and the position information. The line-of-sight area determination module is further configured to: count the duration of the driver's gaze on each area of ​​the windshield based on the driving observation angle; and determine the target gaze area based on the gaze duration; and determine the driver's target line-of-sight area on the windshield based on the location information and the target gaze area. The cleaning module is also used to determine the degree of impact of foreign object information in the target visual area on the driver's vision based on the second analysis result, wherein the second analysis result is the position of the foreign object in the target visual area and the area it occupies; and to adjust the swing angle, swing frequency and swing time of the windshield wipers when cleaning the windshield according to the degree of visual impact.

Images