A method and system for controlling a wiper of a commercial vehicle

Abstract

The application discloses a rain screen control method and system for a commercial vehicle. The method comprises: a driving recorder acquiring scene images in a rainy driving scene and uploading them to the cloud; the cloud processing each image frame in the scene images, extracting raindrop density, wet area proportion, rain gear density and rain screen action rate for feature fusion, and inputting the fused features into a trained neural network model for rain amount prediction; based on the driving speed and acceleration of the target vehicle in each image frame, the rain amount result is corrected; the rain amount correction results of all image frames are accumulated to obtain the rain intensity, and are issued to the driving recorder of the target vehicle; the driving recorder finds a rain screen control strategy matched with the rain intensity according to the rain intensity, and controls the rain screen swing frequency and the rain screen adsorption strength based on the found rain screen control strategy. The application can improve the prediction accuracy of rain intensity, accurately match the rain screen control strategy, and improve the driving safety in rainy scenes.

Description

Technical Field

[0001] This application relates to the field of intelligent control system technology for automobiles, specifically to the fields of image detection and recognition and wiper control, and particularly to a wiper control method and system for commercial vehicles. Background Technology

[0002] Existing sunlight and rain sensors suffer from low detection accuracy, malfunction, and false alarms due to factors such as pollutant coverage, low temperatures, snow, and uneven snow melting. These sensors are no longer adequate for the automated vehicle control requirements in complex weather conditions, particularly for windshield wiper control. Therefore, it is necessary to design a new wiper control scheme to improve its resistance to interference and detection reliability under harsh conditions with varying rainfall levels, ensuring stable operation of the automatic wiper function and further enhancing driving safety and convenience. Summary of the Invention

[0003] This application provides a windshield wiper control method and system for commercial vehicles to solve the problem of inaccurate windshield wiper control caused by inaccurate detection or malfunction of existing sunlight and rain sensors, which poses a potential danger to vehicle driving safety.

[0004] The technical solution is as follows:

[0005] Firstly, a method for controlling windshield wipers in a commercial vehicle is provided, including:

[0006] The dashcam acquires images of the scene in front of the target vehicle and uploads them to the cloud;

[0007] The cloud segment and precisely outlines the raindrops in each image frame of the received scene image, uses a three-dimensional convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames in the scene image that do not meet the preset raindrop size and / or raindrop movement duration conditions.

[0008] The cloud-based system extracts raindrops based on segmentation from all retained image frames, determines the raindrop density in each image frame, extracts the total pixel area of ​​the ground and the pixel area of ​​the wet ground in each image frame, and calculates the proportion of the wet area; identifies rain gears and pedestrians in each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame and calculates the windshield wiper activation rate.

[0009] The cloud-based system fuses the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame, and inputs the fused features of each image frame into a pre-trained neural network model for rainfall prediction.

[0010] The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; and accumulates the rainfall correction results for each image frame to obtain the rainfall intensity in the scene image, and sends it to the dashcam of the target vehicle.

[0011] The dashcam searches for a wiper control strategy that matches the current rainfall intensity from its local storage, and controls the wiper oscillation frequency and wiper suction intensity based on the found wiper control strategy.

[0012] In one possible implementation, the cloud determines the raindrop density in each image frame based on the raindrops extracted from all retained image frames, specifically including:

[0013] The cloud-based system determines the pixel area of ​​raindrops in each image frame based on the raindrops extracted from all retained image frames.

[0014] The effective area of ​​all pixels in each image frame is counted, and the ratio of the pixel area of ​​the raindrop to the effective area of ​​all pixels is taken as the raindrop density in each image frame.

[0015] In one possible implementation, the cloud identifies rain gear and pedestrians from each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers activated and the total number of vehicles from each image frame, and calculates the wiper activation rate; specifically including:

[0016] The cloud-based system identifies the number of rain gears and pedestrians in each image frame; the rain gears include raincoats and umbrellas; the ratio of the number of rain gears to the number of pedestrians is used as the rain gear density.

[0017] The number of vehicles with windshield wipers activated and the total number of vehicles are identified in each image frame. The ratio of the number of vehicles with windshield wipers activated to the total number of vehicles is used as the wiper activation rate.

[0018] In one possible implementation, the cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration, specifically including:

[0019] The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and calculates the rain speed and rain direction based on the raindrops in adjacent image frames;

[0020] For a single image frame, the ratio of rain speed to driving speed is multiplied by the target vehicle's acceleration as a correction factor. The corrected rainfall result is the predicted rainfall result plus the predicted rainfall result multiplied by the correction factor.

[0021] In one possible implementation, the dashcam locally stores multiple rainfall intensity levels, each rainfall intensity level corresponding to a wiper control strategy; the method further includes:

[0022] The dashcam adjusts the rainfall intensity range corresponding to the rainfall intensity level based on the driver's input, and / or adjusts the wiper oscillation frequency and / or wiper adsorption intensity in the wiper control strategy to flexibly adapt to the driver's personalized needs.

[0023] Secondly, a windshield wiper control system for a commercial vehicle is provided, comprising: a target vehicle, a driving recorder and windshield wipers installed on the target vehicle, and a cloud platform that establishes a communication connection with the target vehicle; wherein...

[0024] The dashcam acquires images of the scene in front of the target vehicle and uploads them to the cloud.

[0025] The cloud segment and precisely outlines the raindrops in each image frame of the received scene image, uses a three-dimensional convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames in the scene image that do not meet the preset raindrop size and / or raindrop movement duration conditions.

[0026] The cloud-based system extracts raindrops based on segmentation from all retained image frames, determines the raindrop density in each image frame, extracts the total pixel area of ​​the ground and the pixel area of ​​the wet ground in each image frame, and calculates the proportion of the wet area; identifies rain gears and pedestrians in each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame and calculates the windshield wiper activation rate.

[0027] The cloud-based system fuses the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame, and inputs the fused features of each image frame into a pre-trained neural network model for rainfall prediction.

[0028] The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; and accumulates the rainfall correction results for each image frame to obtain the rainfall intensity in the scene image, and sends it to the dashcam of the target vehicle.

[0029] The dashcam searches for a wiper control strategy that matches the current rainfall intensity from its local storage, and controls the wiper oscillation frequency and wiper suction intensity based on the found wiper control strategy.

[0030] Thirdly, an electronic device is provided, comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0034] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the aspects described above and any possible implementation thereof.

[0035] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0036] Sixthly, a commercial vehicle is provided, including the electronic equipment described above.

[0037] The beneficial effects of the technical solution provided in this application include at least the following:

[0038] As can be seen from the above technical solution, in this embodiment, a dashcam is used instead of a sunlight and rain sensor to acquire scene images of a rainy driving scenario and upload them to the cloud. The cloud processes each image frame in the scene image, extracts raindrop density, wet area ratio, rain gear density, and wiper action rate for feature fusion, and inputs the fused features of each image frame into a trained neural network model for rainfall prediction. Then, based on the driving speed and acceleration of the target vehicle in each image frame, the rainfall result is corrected. The rainfall correction results of each image frame are accumulated to obtain the rainfall intensity in the scene image and sent to the dashcam of the target vehicle. The dashcam searches for a wiper control strategy that matches the current rainfall intensity locally based on the rainfall intensity, and controls the wiper oscillation frequency and wiper adsorption intensity based on the found wiper control strategy. This application can obtain rainfall intensity by collecting scene images from a dashcam, and then correct the rainfall intensity using driving speed and acceleration to obtain accurate rainfall intensity data. The rainfall intensity is then used to accurately match the wiper control side, and the wiper swing frequency and wiper adsorption intensity are reasonably matched to control the wiper swing, reduce the impact on the driver's vision, and improve driving safety in rainy weather.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the scenario architecture applicable to the wiper control scheme for commercial vehicles provided in this application embodiment.

[0042] Figure 2 This is a schematic diagram of the steps of a windshield wiper control method for a commercial vehicle provided in an embodiment of this application.

[0043] Figure 3 This is a schematic diagram of the wiper control process for a commercial vehicle provided in an embodiment of this application.

[0044] Figure 4 This is a structural block diagram of the windshield wiper control system for a commercial vehicle provided in an embodiment of this application.

[0045] Figure 5 This is a block diagram of an electronic device used to implement the windshield wiper control method for a commercial vehicle according to embodiments of this application. Detailed Implementation

[0046] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0047] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0048] It should be noted that the terminal devices involved in the embodiments of this application may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0049] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0050] In view of the problem that inaccurate detection or malfunction of existing sunlight and rain sensors leads to inaccurate wiper control, posing a potential danger to vehicle driving safety, this application proposes a wiper control scheme for commercial vehicles. The main inventive concept is as follows: a dashcam is used to replace the sunlight and rain sensor for detection, acquiring scene images of rainy driving scenarios and uploading them to the cloud. The cloud processes each image frame, extracting raindrop density, wet area ratio, rain gear density, and wiper action rate for feature fusion. The fused features of each image frame are then input into a trained neural network model for rainfall prediction. Subsequently, based on the target vehicle's driving speed and acceleration in each image frame, the rainfall results are corrected. The corrected rainfall results for each image frame are accumulated to obtain the rainfall intensity in the scene image, which is then sent to the dashcam of the target vehicle. The dashcam, based on the rainfall intensity, searches locally for a wiper control strategy matching the current rainfall intensity and controls the wiper oscillation frequency and wiper adsorption intensity based on the found wiper control strategy. This application can obtain rainfall intensity by collecting scene images from a dashcam, and then correct the rainfall intensity using driving speed and acceleration to obtain accurate rainfall intensity data. The rainfall intensity is then used to accurately match the wiper control side, and the wiper swing frequency and wiper adsorption intensity are reasonably matched to control the wiper swing, reduce the impact on the driver's vision, and improve driving safety in rainy weather.

[0051] Reference Figure 1 The diagram shown illustrates the scenario architecture applicable to the wiper control scheme for commercial vehicles provided in this application embodiment. This scenario architecture includes one or more target vehicles 101 and a cloud platform 102. The target vehicles 101 can establish a communication connection with the cloud platform 102, uploading their own vehicle data to the cloud platform 102 for processing, and simultaneously receiving data sent from the cloud platform 102.

[0052] like Figure 1As shown, a dashcam 1011 is installed on the front or rear of the target vehicle 101, and a windshield wiper 1012 is installed on the windshield of the target vehicle 101. The dashcam 1011 can be any existing image acquisition device capable of capturing and recording scene images; it is used to collect real-time scene images of the target vehicle 101 in the driving environment, especially in rainy weather; and uploads these scene images to the cloud 102 for processing. The windshield wiper 1012 can be an intelligent wiper device with adjustable adsorption capacity. It receives control commands from the dashcam 1011 through a built-in control module, analyzes the commands, and manipulates the friction between its own wiper surface and the windshield, thereby adjusting the adsorption capacity between the two.

[0053] Cloud102 can be configured as a cloud service with distributed or edge computing capabilities, providing rainfall intensity or other computing services to a large number of commercial vehicles through efficient and powerful computing capabilities.

[0054] Reference Figure 2 The diagram shown illustrates the steps of a windshield wiper control method for a commercial vehicle according to an embodiment of this application. The windshield wiper control method may include the following steps:

[0055] Step 202: The dashcam acquires scene images of the area in front of the target vehicle and uploads them to the cloud.

[0056] In this application, the solar rain sensor used to detect rainfall is replaced by a dashcam, which captures images of the scene in front of the target vehicle and then uploads them to the cloud for rainfall intensity calculation. Therefore, a solar rain sensor is not needed, avoiding situations where rainfall cannot be accurately detected due to severe rain or obstruction.

[0057] Step 204: The cloud segment and precisely outlines the raindrops in each image frame of the received scene image, uses a three-dimensional convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames in the scene image that do not meet the preset raindrop size and / or raindrop movement duration conditions.

[0058] In this application's solution, UET can be used for precise raindrop segmentation and outlining in a single frame, and a 3D CNN model is used for video analysis to capture the dynamic movement paths of raindrops. Based on a preset raindrop size, raindrop images smaller than the preset size are discarded, as these are considered misjudged data. Simultaneously, the captured dynamic raindrops are monitored to determine if their movement duration is less than a preset duration. For example, if the preset duration is set to 1 second, then if the movement duration is less than 1 second, it is considered misjudged data and the image is discarded.

[0059] Step 206: The cloud-based system extracts raindrops based on the segmentation of all retained image frames, determines the raindrop density in each image frame, extracts the total pixel area of ​​the ground and the pixel area of ​​the wet ground in each image frame, and calculates the proportion of the wet area; identifies rain gears and pedestrians in each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame and calculates the windshield wiper operation rate.

[0060] Optionally, when determining the raindrop density in each image frame based on the raindrops extracted from all retained image frames, the cloud can specifically determine the pixel area of ​​the raindrops in each image frame based on the raindrops extracted from all retained image frames; count the total effective area of ​​all pixels in each image frame, and use the ratio of the pixel area of ​​the raindrops to the total effective area of ​​all pixels as the raindrop density in each image frame.

[0061] In this application, the cloud performs image segmentation on the raindrops in each of the retained image frames, extracts the raindrop pixel images, and then calculates the pixel area of ​​the raindrops in each image frame. Simultaneously, it calculates the total effective area of ​​all pixels in each image frame. Since the total pixel area of ​​each image frame is generally approximately equal, the total effective area of ​​only one image frame can be calculated as the total effective area of ​​all image frames. Then, the ratio of the pixel area of ​​the raindrops in each image frame to the total effective area of ​​all pixels is used as the raindrop density in each image frame.

[0062] Furthermore, the cloud can also extract the total pixel area of ​​the ground and the pixel area of ​​the wet ground from each image frame based on image recognition technology. In this way, for each image frame, the ratio of the total pixel area of ​​the ground to the pixel area of ​​the wet ground can be used as the proportion of the wet area in each image frame.

[0063] Optionally, the cloud identifies rain gear and pedestrians in each image frame and calculates rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame and calculates the wiper operation rate; specifically, it can identify the number of rain gears and pedestrians in each image frame; wherein, rain gear includes: raincoats and umbrellas; the ratio of the number of rain gears to the number of pedestrians is used as rain gear density; and it identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame, and uses the ratio of the number of vehicles with windshield wipers on to the total number of vehicles as wiper operation rate.

[0064] In practical implementation, the cloud can use locally configured image recognition technology to identify pedestrians and those wearing raincoats or holding umbrellas from each image frame, and count the number of pedestrians and the number of rain gear such as raincoats and umbrellas. The ratio of the number of rain gear to the number of pedestrians is used as the rain gear density. Simultaneously, it can also use image recognition technology to identify all vehicles from each image frame, and from all vehicles, identify those with windshield wipers on, and count the number of vehicles with wipers on and the total number of vehicles to calculate the ratio of the number of vehicles with wipers on to the total number of vehicles as the wiper activation rate.

[0065] Therefore, through step 206, based on cloud-based image recognition technology and configured algorithms, the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame can be determined, thereby accurately extracting the relevant features required for predicting rainfall intensity and providing reliable data support for subsequent accurate rainfall intensity prediction.

[0066] Step 208: The cloud platform fuses the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame, and inputs the fused features of each image frame into a trained neural network model for rainfall prediction.

[0067] In this application, the neural network model involved can be a feedforward neural network (FNN), a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder (AE), a generative adversarial network (GAN), etc. Taking CNN as an example, feature extraction can be performed on historical image frames in historical scene images based on the CNN network. That is, the raindrop density, wet area ratio, rain gear density, and wiper action rate of each historical image frame can be extracted as sample features and input into the subsequent network of the CNN model. At the same time, the approximate rainfall intensity of each historical image frame can be obtained from the weather forecast or rainfall monitoring measurement at that time and used as sample labels to input into the CNN model. Then, the model is repeatedly trained to obtain a converged rainfall prediction model, that is, a trained neural network model.

[0068] It should be understood that during the model training phase, the methods for obtaining the raindrop density, wet area ratio, rain gear density, and wiper activation rate for each historical image frame are the same as those for obtaining the raindrop density, wet area ratio, rain gear density, and wiper activation rate for the real-time image frames in the prediction phase of step 206. Therefore, they will not be repeated here.

[0069] In step 208, the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame can be fused using global pooling or self-attention mechanisms to obtain the fused features for each frame. Then, the fused features of all image frames are input into a pre-trained neural network model to predict rainfall for each image frame.

[0070] Step 210: The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; and accumulates the rainfall correction results for each image frame to obtain the rainfall intensity in the scene image, and sends it to the dashcam of the target vehicle.

[0071] Optionally, when the cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall result for each image frame based on the driving speed and acceleration, it can specifically acquire the driving speed and acceleration of the target vehicle in each image frame of the scene image, and simultaneously calculate the rain speed and rain direction based on the raindrops in adjacent image frames; for a single image frame, the ratio of rain speed to driving speed is multiplied by the acceleration of the target vehicle as a correction factor, and the corrected rainfall result is the predicted rainfall result plus the predicted rainfall result multiplied by the correction factor.

[0072] Specifically, based on the time information of each frame in the scene image, the target vehicle's speed and acceleration in each frame can be found. The speed and acceleration can be determined based on the target vehicle's overall driving information. Simultaneously, the positions of one or more raindrops can be marked in each image frame. Then, based on the raindrop positions in adjacent image frames, the rain speed is calculated; and the direction of rainfall is determined from the direction of raindrop fall in the image frames. For each image frame, a correction factor is determined based on the rain speed, speed, and acceleration: the ratio of rain speed to speed multiplied by the target vehicle's acceleration. Thus, each image frame can have a correction factor determined, and the corrected rainfall result for each image frame is: the predicted rainfall result + the predicted rainfall result multiplied by the correction factor.

[0073] Furthermore, the rainfall correction results of all image frames are summed to obtain the rainfall intensity in the scene image, and the cloud sends the rainfall intensity to the dashcam of the target vehicle.

[0074] Step 212: The dashcam searches for a wiper control strategy that matches the current rainfall intensity from the local storage, and controls the wiper oscillation frequency and wiper suction intensity based on the found wiper control strategy.

[0075] Optionally, the dashcam locally stores multiple rainfall intensity levels, each corresponding to a wiper control strategy; then the dashcam can also adjust the rainfall intensity range corresponding to the rainfall intensity level based on the driver's input, and / or adjust the wiper oscillation frequency and / or wiper adsorption intensity in the wiper control strategy, so as to flexibly adapt to the driver's personalized needs.

[0076] Specifically, this application's solution can also add a vehicle group information sharing mechanism. Based on the sales information of the target vehicle model, all target vehicles of that model are grouped, and the target vehicles within each group share information such as rainfall intensity. When the dashcam of a target vehicle in a group malfunctions or the video analysis is abnormal, the TBOX receives the dashcam data and requests rainfall and sunlight data from the cloud. The cloud, based on the location of the vehicles in the group, sends cloud data of vehicles within a certain distance ahead of it. The TBOX receives this data and uses it as the current data to adjust the wipers and automatic headlights, adding a failure redundancy mechanism for vehicles behind.

[0077] Reference Figure 3 The diagram shown is a schematic of the windshield wiper control process for a commercial vehicle according to an embodiment of this application. This mainly involves the interaction between the commercial vehicle and a cloud server, specifically the interaction between the vehicle's TBOX and the cloud server. In addition to the TBOX, the system also includes a dashcam and a wiper controller.

[0078] The specific process is as follows: Figure 3 As shown, firstly, the dashcam acquires scene images. The dashcam filters out raindrops, and then uploads the filtered scene images to the cloud server via the TBOX. The cloud server determines the raindrop density, wet area percentage, rain gear density, and wiper operation rate for each image frame in the scene image, and then performs feature extraction and fusion to obtain fused features for each image frame. Based on the fused features, a prediction model is used to predict rainfall for each image frame. The rainfall prediction is then corrected, and the rainfall correction results for all image frames are accumulated to obtain the rainfall intensity. The rainfall intensity of the scene image is then sent to the commercial vehicle's TBOX, which forwards it to the dashcam. The dashcam matches a wiper control strategy based on the rainfall intensity and controls the wiper operation using the wiper oscillation frequency and wiper suction strength within the strategy. Therefore, by leveraging the participation of the dashcam and the cloud server, the accuracy of rainfall intensity prediction can be improved, ultimately matching a suitable wiper control strategy that aligns with the driver's driving habits and enhances driving safety.

[0079] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0080] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0081] Figure 4 This application illustrates a structural block diagram of a windshield wiper control system for a commercial vehicle according to one embodiment of the present application, as shown below. Figure 4 As shown. The wiper control system 400 of the commercial vehicle in this embodiment may include a target vehicle 401, a driving recorder 4011 and a wiper 4012 installed on the target vehicle 401, and a cloud 402 that establishes a communication connection with the target vehicle 401. The dashcam 4011 acquires scene images of the area in front of the target vehicle 401 and uploads them to the cloud 402. The cloud 402 segments and precisely outlines the raindrops in each image frame of the received scene image, uses a 3D convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames that do not meet preset raindrop size and / or raindrop movement duration conditions. Based on the segmented and extracted raindrops from all retained image frames, the cloud 402 determines the raindrop density in each image frame; it also extracts the total pixel area of ​​the ground and the pixel area of ​​the wet ground area in each image frame, and calculates the proportion of the wet area; it identifies rain gear and pedestrians in each image frame and calculates the rain gear density; and it identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame, and calculates the wiper action. The cloud 402 fuses the raindrop density, wet area ratio, rain gear density, and wiper action rate of each image frame, and inputs the fused features of each image frame into a trained neural network model for rainfall prediction; the cloud 402 obtains the driving speed and acceleration of the target vehicle 401 in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; and accumulates the rainfall correction results of each image frame to obtain the rainfall intensity in the scene image, and sends it to the dashcam 4011 of the target vehicle; the dashcam 4011 searches for a wiper control strategy that matches the current rainfall intensity from its local storage, and controls the oscillation frequency and wiper adsorption intensity of the wiper 4012 based on the found wiper control strategy.

[0082] Optionally, in one possible implementation of this embodiment, when the cloud 402 determines the raindrop density in each image frame based on the raindrops extracted from all retained image frames, it is specifically used to determine the pixel area of ​​the raindrops in each image frame based on the raindrops extracted from all retained image frames; count the total effective area of ​​all pixels in each image frame, and use the ratio of the pixel area of ​​the raindrops to the total effective area of ​​all pixels as the raindrop density in each image frame.

[0083] Optionally, in one possible implementation of this embodiment, when the cloud 402 identifies rain gear and pedestrians from each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles from each image frame and calculates the windshield wiper operation rate; specifically, it is used to identify the number of rain gears and pedestrians from each image frame; wherein, the rain gear includes: raincoats and umbrellas; the ratio of the number of rain gears to the number of pedestrians is used as the rain gear density; and the ratio of the number of vehicles with windshield wipers on to the total number of vehicles is used as the windshield wiper operation rate.

[0084] Optionally, in one possible implementation of this embodiment, when the cloud 402 acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall result for each image frame based on the driving speed and acceleration, it specifically acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and calculates the rain speed and rain direction based on the raindrops in adjacent image frames; for a single image frame, the ratio of rain speed to driving speed is multiplied by the acceleration of the target vehicle as a correction factor, and the corrected rainfall result is the predicted rainfall result plus the predicted rainfall result multiplied by the correction factor.

[0085] Optionally, in one possible implementation of this embodiment, the dashcam 4011 locally stores multiple rainfall intensity levels, each rainfall intensity level corresponding to a wiper control strategy; the dashcam 4011 can also adjust the rainfall intensity range corresponding to the rainfall intensity level based on the driver's input operation, and / or adjust the wiper oscillation frequency and / or wiper adsorption intensity in the wiper control strategy, so as to flexibly adapt to the driver's personalized needs.

[0086] In this embodiment, a dashcam can be used instead of a sunlight and rain sensor to acquire scene images of a rainy driving scenario and upload them to the cloud. The cloud processes each image frame in the scene image, extracting raindrop density, wet area ratio, rain gear density, and wiper operation rate for feature fusion. The fused features of each image frame are then input into a pre-trained neural network model for rainfall prediction. Subsequently, the rainfall results are corrected based on the target vehicle's driving speed and acceleration in each image frame. The rainfall correction results of each image frame are accumulated to obtain the rainfall intensity in the scene image and sent to the dashcam of the target vehicle. The dashcam searches locally for a wiper control strategy that matches the current rainfall intensity based on the rainfall intensity and controls the wiper oscillation frequency and wiper adsorption intensity based on the found wiper control strategy. This application can obtain rainfall intensity by collecting scene images from a dashcam, and then correct the rainfall intensity using driving speed and acceleration to obtain accurate rainfall intensity data. The rainfall intensity is then used to accurately match the wiper control side, and the wiper swing frequency and wiper adsorption intensity are reasonably matched to control the wiper swing, reduce the impact on the driver's vision, and improve driving safety in rainy weather.

[0087] One embodiment of this application provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the windshield wiper control method for a commercial vehicle as described above.

[0088] One embodiment of this application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the windshield wiper control method for a commercial vehicle as described above.

[0089] One embodiment of this application provides a commercial vehicle including the electronic devices described above. Specifically, the commercial vehicle may be an autonomous driving vehicle, such as a Level 2 or higher autonomous driving vehicle.

[0090] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0091] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0092] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0093] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0094] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for controlling windshield wipers in a blind commercial vehicle. For example, in some embodiments, the method for controlling windshield wipers in a commercial vehicle can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method for controlling windshield wipers in a commercial vehicle described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured by any other suitable means (e.g., by means of firmware) to perform a method of windshield wiper control for a commercial vehicle.

[0095] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0096] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0100] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0101] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for controlling windshield wipers in a commercial vehicle, characterized in that, include: The dashcam acquires images of the scene in front of the target vehicle and uploads them to the cloud; The cloud segment and precisely outlines the raindrops in each image frame of the received scene image, uses a three-dimensional convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames in the scene image that do not meet the preset raindrop size and / or raindrop movement duration conditions. The cloud-based system determines the raindrop density in each image frame based on the raindrops extracted from all retained image frames. Additionally, extract the total pixel area of ​​the ground and the pixel area of ​​the wet ground in each image frame, and calculate the proportion of the wet area; Identify rain gear and pedestrians in each image frame and calculate rain gear density; And identify the number of vehicles with windshield wipers turned on and the total number of vehicles from each image frame, and calculate the wiper activation rate; The cloud-based system fuses the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame, and inputs the fused features of each image frame into a pre-trained neural network model for rainfall prediction. The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; In addition, the rainfall correction results of each image frame are accumulated to obtain the rainfall intensity in the scene image, and then sent to the dashcam of the target vehicle. The dashcam searches for a wiper control strategy that matches the current rainfall intensity from its local storage, and controls the wiper oscillation frequency and wiper suction intensity based on the found wiper control strategy.

2. The method as described in claim 1, characterized in that, The cloud-based system determines the raindrop density in each image frame based on the segmented raindrops extracted from all retained image frames, specifically including: The cloud-based system determines the pixel area of ​​raindrops in each image frame based on the raindrops extracted from all retained image frames. The effective area of ​​all pixels in each image frame is counted, and the ratio of the pixel area of ​​the raindrop to the effective area of ​​all pixels is taken as the raindrop density in each image frame.

3. The method as described in claim 1, characterized in that, The cloud-based system identifies rain gear and pedestrians from each image frame and calculates the density of rain gear. And it identifies the number of vehicles with windshield wipers activated and the total number of vehicles in each image frame, and calculates the wiper activation rate; specifically including: The cloud-based system identifies the number of rain gears and pedestrians in each image frame; the rain gears include raincoats and umbrellas; the ratio of the number of rain gears to the number of pedestrians is used as the rain gear density. The number of vehicles with windshield wipers activated and the total number of vehicles are identified in each image frame. The ratio of the number of vehicles with windshield wipers activated to the total number of vehicles is used as the wiper activation rate.

4. The method as described in claim 3, characterized in that, The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration, specifically including: The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and calculates the rain speed and rain direction based on the raindrops in adjacent image frames; For a single image frame, the ratio of rain speed to driving speed is multiplied by the target vehicle's acceleration as a correction factor. The corrected rainfall result is the predicted rainfall result plus the predicted rainfall result multiplied by the correction factor.

5. The method as described in claim 3, characterized in that, The dashcam locally stores multiple rainfall intensity levels, each corresponding to a wiper control strategy; the method further includes: The dashcam adjusts the rainfall intensity range corresponding to the rainfall intensity level based on the driver's input, and / or adjusts the wiper oscillation frequency and / or wiper adsorption intensity in the wiper control strategy to flexibly adapt to the driver's personalized needs.

6. A windshield wiper control system for a commercial vehicle, characterized in that, include: The target vehicle, the dashcam and windshield wipers installed on the target vehicle, and the cloud platform that establishes a communication connection with the target vehicle; wherein... The dashcam acquires images of the scene in front of the target vehicle and uploads them to the cloud. The cloud segment and precisely outlines the raindrops in each image frame of the received scene image, uses a three-dimensional convolutional neural network for image analysis, captures the dynamic raindrop paths in the scene image, and removes image frames in the scene image that do not meet the preset raindrop size and / or raindrop movement duration conditions. The cloud-based system extracts raindrops based on segmentation from all retained image frames, determines the raindrop density in each image frame, extracts the total pixel area of ​​the ground and the pixel area of ​​the wet ground in each image frame, and calculates the proportion of the wet area; identifies rain gears and pedestrians in each image frame and calculates the rain gear density; and identifies the number of vehicles with windshield wipers on and the total number of vehicles in each image frame and calculates the windshield wiper activation rate. The cloud-based system fuses the raindrop density, wet area ratio, rain gear density, and wiper operation rate of each image frame, and inputs the fused features of each image frame into a pre-trained neural network model for rainfall prediction. The cloud acquires the driving speed and acceleration of the target vehicle in each image frame of the scene image, and corrects the predicted rainfall results for each image frame based on the driving speed and acceleration; and accumulates the rainfall correction results for each image frame to obtain the rainfall intensity in the scene image, and sends it to the dashcam of the target vehicle. The dashcam searches for a wiper control strategy that matches the current rainfall intensity from its local storage, and controls the wiper oscillation frequency and wiper suction intensity based on the found wiper control strategy.

7. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

9. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

10. A commercial vehicle including the electronic equipment as claimed in claim 7.

Images