Windscreen wiper control method, device, equipment, medium, product and vehicle

Abstract

The invention relates to a windscreen wiper control method and device, equipment, a medium, a product and a vehicle. A target image shot for at least part of a vehicle body of the vehicle is obtained; acquiring detection information collected for the environment where the vehicle is located; performing rainfall detection according to the target image and the detection information to obtain rainfall information; and controlling the windscreen wiper of the vehicle according to the rainfall information. Therefore, the windscreen wiper is controlled by combining the images shot for at least part of the vehicle body of the vehicle and the rainfall information obtained by carrying out rainfall detection on the detection information collected for the environment where the vehicle is located, and therefore the accuracy of controlling the windscreen wiper to work is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a windshield wiper control method, device, equipment, medium, product, and vehicle. Background Technology

[0002] Currently, windshield wiper control in vehicles is generally achieved through rain sensors. Specifically, rain sensors installed on the vehicle detect the amount of rainfall and control the vehicle's windshield wipers to operate based on the detected rainfall amount.

[0003] However, due to issues such as the data sensed by rain sensors being easily interfered with by noise and the small sensing range, the accuracy of controlling the vehicle's windshield wipers based on the amount of rainfall detected by the sensors is relatively low. Summary of the Invention

[0004] This application provides a windshield wiper control method, device, electronic device, storage medium, and computer program product. By combining images captured of at least a portion of the vehicle body with rainfall information obtained from rainfall detection based on the vehicle's environment, the windshield wipers can be controlled, thereby improving the accuracy of windshield wiper operation.

[0005] This application provides a windshield wiper control method, including:

[0006] Acquire target images of at least a portion of the vehicle body;

[0007] Acquire detection information about the environment in which the vehicle is located;

[0008] Rainfall information is obtained by performing rainfall detection based on the target image and the detection information.

[0009] The vehicle's windshield wipers are controlled based on the rainfall information.

[0010] Accordingly, embodiments of this application provide a windshield wiper control device, including:

[0011] The target image acquisition module is used to acquire a target image of at least a portion of the vehicle body.

[0012] The detection information acquisition module is used to acquire detection information collected about the environment in which the vehicle is located;

[0013] A rainfall detection module is used to detect rainfall based on the target image and the detection information to obtain rainfall information;

[0014] The control module is used to control the windshield wipers of the vehicle based on the rainfall information.

[0015] Furthermore, this application also provides an electronic device, including one or more processors and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the wiper control method provided in this application.

[0016] Furthermore, this application embodiment also provides a storage medium storing a computer program. When the computer program is run on a processor, the computer program is used to cause the processor to execute any of the wiper control methods provided in this application embodiment.

[0017] Furthermore, this application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement any of the windshield wiper control methods provided in this application.

[0018] Furthermore, this application also provides a vehicle that includes the aforementioned electronic equipment.

[0019] In this embodiment, a target image of at least a portion of the vehicle body is acquired; detection information of the vehicle's environment is acquired; rainfall information is obtained by performing rainfall detection based on the target image and the detection information; and the vehicle's windshield wipers are controlled based on the rainfall information. Thus, by combining the image of at least a portion of the vehicle body with the rainfall information obtained from rainfall detection using the detection information of the vehicle's environment, the windshield wipers are controlled, thereby improving the accuracy of windshield wiper operation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an implementation environment scenario for the wiper control method provided in this application embodiment;

[0022] Figure 2 This is a schematic flowchart of a windshield wiper control method provided in one embodiment of this application;

[0023] Figure 3 This is a schematic flowchart of a windshield wiper control method provided in one embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a wiper control device provided in one embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] Furthermore, in the embodiments of this application, "multiple" refers to two or more. The terms "first" and "second," etc., in the embodiments of this application are used for distinguishing descriptions and should not be construed as implying relative importance.

[0028] This application provides a windshield wiper control method, apparatus, electronic device, storage medium, computer program product, and vehicle. The windshield wiper control apparatus can be integrated into an electronic device, which can be a server, a terminal, or other similar device.

[0029] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0030] The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0031] Please see Figure 1 Taking the integration of windshield wiper control devices into electronic devices as an example, Figure 1 This is a schematic diagram of an implementation scenario of the windshield wiper control method provided in this application. The electronic device can be a terminal device, which acquires a target image of at least part of the vehicle body; acquires detection information collected on the environment in which the vehicle is located; performs rainfall detection based on the target image and the detection information to obtain rainfall information; and controls the vehicle's windshield wipers based on the rainfall information.

[0032] It should be noted that, Figure 1The illustrated scenario of the wiper control method is merely an example. The implementation environment of the wiper control method described in this application is for the purpose of more clearly illustrating the technical solution of this application and does not constitute a limitation on the technical solution provided in this application. Those skilled in the art will understand that with the evolution of data processing and the emergence of new business scenarios, the technical solution provided in this application is also applicable to similar technical problems.

[0033] The solutions provided in this application are specifically illustrated through the following embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0034] This embodiment will be described from the perspective of a windshield wiper control device, which can be integrated into an electronic device, such as a terminal and / or a server, and this application does not impose any limitations on it.

[0035] Please see Figure 2 , Figure 2 This is a schematic flowchart of a windshield wiper control method according to an embodiment of this application. The windshield wiper control method may include the following steps S101 to S104:

[0036] S101. Acquire a target image of at least part of the vehicle body.

[0037] In this context, "vehicle" refers to a vehicle equipped with windshield wipers. The type of vehicle can be adjusted according to actual circumstances, and this application embodiment does not impose any limitations. For example, a vehicle can be a car, a train, a tram, etc.

[0038] In this context, "vehicle body" refers to a component of a vehicle. The specific details of the vehicle body can be adjusted according to actual circumstances, and this application's embodiments do not impose limitations. For example, the vehicle body may be the vehicle's windows. Another example is the vehicle's windshield. Yet another example is the vehicle's roof.

[0039] The target image refers to an image captured of the vehicle body that can be used for rainfall detection. The specific content of the target image can be adaptively adjusted according to the vehicle body, and this application embodiment does not impose limitations. For example, when the vehicle body refers to a vehicle window, the target image can be an image captured from any window of the vehicle. As another example, when the vehicle body refers to the vehicle roof, the target image can be an image captured from the vehicle roof.

[0040] S102. Obtain detection information about the environment in which the vehicle is located.

[0041] Among them, the detection information indicator can be used for rainfall detection, based on the information collected about the environment in which the vehicle is located.

[0042] The detection information can be of various types, and its specific content can be adjusted according to the actual situation. This application does not impose any limitations on the embodiments. For example, the detection information can be rainfall information, or it can be rain sound information.

[0043] The detection information can be acquired through detection equipment, which may or may not be installed on a vehicle. There are various types of detection equipment, and the specific content of the detection equipment can be adjusted according to the actual situation; this application does not impose any limitations. For example, when the detection information is rainfall information, the detection equipment can be a rain sensor, radar, or a device composed of a rain sensor and radar. As another example, when the detection information is rain sound information, the detection equipment can be a sound sensor.

[0044] S103. Rainfall detection is performed based on the target image and detection information to obtain rainfall information.

[0045] Rainfall information refers to information obtained by combining images taken of at least part of a vehicle body with detection information collected about the environment in which the vehicle is located.

[0046] Rainfall information is used to assess rainfall conditions in the vehicle's environment. Rainfall information can be categorized using various criteria, and its specific content can be adjusted according to actual circumstances; this application's embodiments do not impose limitations. For example, rainfall information can be divided into multiple stages, such as no rain, light rain, moderate rain, and heavy rain. Another example is that rainfall information can be categorized by specific rainfall amounts, such as rainfall of 5mm, where 5mm indicates the accumulated water depth of 5mm over a period of time.

[0047] S104. Control the vehicle's windshield wipers based on rainfall information.

[0048] Specifically, there are various ways to control the windshield wipers of a vehicle based on rainfall information, and these methods can be adjusted according to the actual situation. This application does not impose any restrictions on these methods.

[0049] For example, based on rainfall information, the system can control the vehicle's windshield wipers to operate, and can also control them to operate at different speeds. Furthermore, it can control the wipers to stop working, or even disable them entirely.

[0050] Therefore, the windshield wiper control method provided in this application involves acquiring a target image of at least a portion of the vehicle body; acquiring detection information collected about the vehicle's environment; performing rainfall detection based on the target image and detection information to obtain rainfall information; and controlling the vehicle's windshield wipers based on the rainfall information. Thus, by combining the image of at least a portion of the vehicle body with the rainfall information obtained from rainfall detection based on the detection information collected about the vehicle's environment, the windshield wipers can be controlled, thereby improving the accuracy of windshield wiper operation.

[0051] In some optional embodiments, the process of obtaining detection information about the vehicle's environment may include: collecting sound information about the vehicle's environment to obtain detection information.

[0052] Specifically, the vehicle's microphone sensor can collect sound information about the vehicle's environment to obtain an audio file, and the detection information can be determined based on the sound information contained in the audio file.

[0053] In some optional embodiments, the above-mentioned detection information is rain sound information from the sound information.

[0054] The process of collecting sound information about the vehicle's environment to obtain detection information may include: collecting sound information about the vehicle's environment; extracting features from the sound signals contained in the sound information to determine sound features; and extracting sound signals from the sound information whose sound features satisfy the characteristics of rain sounds to obtain rain sound information.

[0055] Before extracting features from the sound signals contained in the sound information and determining the sound features, the sound information can be preprocessed. The specific preprocessing operations can be adjusted according to the actual situation, and this application embodiment does not impose any limitations. For example, noise filtering can be performed on the sound information to filter out background noise other than rain sounds. Another example is filtering out non-essential information (such as human voices or vibrations of vehicle parts) from the sound information. Yet another example is performing frame segmentation on the sound information to enhance the audio quality of high-frequency sound signals, etc.

[0056] There are various methods for feature extraction, and the specific method can be adjusted according to the actual situation. This application does not impose any limitations on these methods. For example, features of the sound signal contained in the sound information can be extracted based on a filter bank (FBank). Alternatively, features of the sound signal contained in the sound information can be extracted based on a power normalized cepstral coefficients (PNCC) method. Another example is the extraction of features of the sound signal contained in the sound information based on Mel-scale frequency cepstral coefficients (MFCC).

[0057] Preferably, this application uses Mel-Cepstral Coefficients to extract the features of the sound signal contained in the sound information.

[0058] Specifically, the process of extracting features from the sound signals contained in the sound information and determining the sound features includes: converting the sound signals contained in the sound information from the linear frequency domain to the Mel frequency domain; and performing cepstral analysis on the sound signals contained in the sound information in the Mel frequency domain to obtain the sound features.

[0059] It should be noted that the basic process for extracting the features of the sound signal based on Mel-frequency cepstral coefficients includes pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel filter bank, logarithmic operation, and Discrete Cosine Transform (DCT). Pre-emphasis processing uses a first-order differential filter to preprocess the speech signal, eliminating redundant information. Framing and windowing divide the speech signal into short time frames and apply window functions for further processing in the frequency domain. The FFT transforms each frame from the time domain to the frequency domain, obtaining a spectrogram. The Mel filter bank maps the spectrogram to the Mel frequency domain; the center frequency of the Mel filter bank is close to the pitch perceived by the human ear. Logarithmic operation takes the logarithm of the Mel spectrum, compressing the dynamic range of the speech signal to a smaller range. Finally, the DCT converts the Mel spectrum into cepstral coefficients, yielding the final sound features.

[0060] By extracting the features of the sound signal contained in the sound information using Mel-Cepstral Coefficients, this processing can obtain more robust sound features and achieve better performance in different audio environments.

[0061] It should be noted that different sounds have different spectral characteristics. Spectral analysis technology can be used to decompose complex sound wave signals into different frequency components. Given the sound characteristics, it is possible to extract sound signals whose sound characteristics satisfy the characteristics of rain sounds from the sound information, and thus obtain rain sound information.

[0062] In some optional embodiments, the process of obtaining rainfall information by detecting rainfall based on target images and detection information may include: determining the target image and rain sound information at the same time based on the capture time of the target image and the acquisition time of the detection information; and performing rainfall detection on the target image and rain sound information at the same time based on a multimodal rainfall detection model to obtain rainfall information.

[0063] Among them, the multimodal rainfall detection model refers to a pre-trained model used for rainfall detection.

[0064] There are various types of multimodal rainfall detection models, and their specific types can be adjusted according to actual conditions. This application does not impose any limitations on these types. For example, a multimodal rainfall detection model may be a deep learning model deployed on vehicle electronic devices, or a semantic model deployed on the vehicle control terminal.

[0065] The training process of the multimodal rainfall detection model can be adjusted according to its specific type, and the embodiments of this application are not limited thereto. For example, the training process of the multimodal rainfall detection model is as follows: collect windshield video and sound information around the car; clean and preprocess the two types of data to keep the image and audio at the same time, so as to obtain useful image and audio data samples in two different forms; classify the samples into light rain, moderate rain and heavy rain using the results fed back by the rain sensor during the collection process, and then supplement it with manual correction, and finally divide the dataset into training samples, validation samples and test samples; input the training samples and validation samples into the initial multimodal model for training and validation to generate candidate rainfall detection models; use test samples to test the candidate models and calculate the accuracy of the models; when the accuracy is greater than a preset threshold, the candidate model is determined as the required multimodal rainfall detection model.

[0066] By using a multimodal rainfall detection model that combines image and sound data for rainfall detection, the inaccurate rainfall detection caused by the small measurement range of existing rain sensors can be effectively avoided. It also solves the problem of image-based wiper control being susceptible to lighting conditions.

[0067] In some optional embodiments, the above-mentioned control of the vehicle's windshield wipers based on rainfall information further includes: acquiring the vehicle's shading status information, which is used to indicate whether the vehicle's environment is sheltered from the rain; and controlling the vehicle's windshield wipers based on the shading status information and rainfall information.

[0068] The occlusion status information includes occlusion status and unoccupied status. The specific content of the occlusion status information can be determined according to the specific environment of the vehicle. For example, the occlusion status information includes indoor and outdoor, where occlusion status indicates indoor and unoccupied status indicates outdoor. As another example, the occlusion status information includes under the parking canopy and outside the parking canopy, where occlusion status indicates under the parking canopy and unoccupied status indicates outside the parking canopy.

[0069] There are various ways to obtain the occlusion status information of a vehicle, and the specific methods can be adjusted according to the actual situation. This application embodiment does not impose any restrictions.

[0070] In some optional embodiments, obtaining the vehicle's occlusion status information can involve obtaining the vehicle's location information and the navigation's location information, and determining the vehicle's occlusion status information based on the vehicle's location information and the navigation's location information.

[0071] In some optional embodiments, the process of obtaining vehicle occlusion status information includes: using a classification model to identify the classification probability of the environment type in which the vehicle is located in the target image based on the target image; and determining the vehicle occlusion status information according to the classification probability of the environment type.

[0072] Among them, the classification model refers to the model that determines the occlusion status information of the vehicle based on the target image.

[0073] The classification probability refers to the probability that the vehicle is in a predetermined environment type. The predetermined environment type is defined by the classification model, which identifies at least two environment types based on the image. For example, the predetermined environment types include indoor and outdoor, and the classification probabilities could be (indoor, 0.7) and (outdoor, 0.3), where (indoor, 0.7) indicates a classification probability of 0.7 for an indoor environment and (outdoor, 0.3) indicates a classification probability of 0.3 for an outdoor environment.

[0074] It should be noted that the training process of the classification model can be adjusted according to its specific type, and this application embodiment does not impose any restrictions. For example, the occlusion status information includes indoor and outdoor, and the classification model is a background classification model. The training process of the background classification model is as follows: acquire image samples, wherein the image samples are the result of data cleaning of the image set generated from the windshield video data; manually classify the images as indoor and outdoor, and divide the dataset into training samples, validation samples, and test samples; use the training samples and validation samples to train and validate the preset initial model to generate candidate background classification models; use the test samples to test the candidate models and calculate the accuracy of the models; when the accuracy is greater than a preset threshold, the candidate model is determined as the required background classification model.

[0075] It should be noted that the process of acquiring image samples can be as follows: a camera installed in front of the vehicle is used to collect video data of the windshield in real time to obtain a video signal that is consistent with the content in the driver's field of vision. The video signal is composed of multiple consecutive frames of image signals. Therefore, the video data can be converted into images of the windshield in real time frame by frame by a driver, raindrops on the windshield can be identified, and preprocessing operations such as adding and reducing noise can be performed on the image based on the raindrops on the windshield image to reduce the impact of noise on the background classification model recognition, thereby obtaining image samples.

[0076] By using a classification model, the number of times the windshield wipers malfunction can be reduced to some extent when the vehicle is in a covered state and there is no rain. This improves the error tolerance of the model recognition and further ensures the accuracy of the windshield wipers starting automatically.

[0077] In some optional embodiments, the process of controlling the vehicle's windshield wipers based on the occlusion status information and rainfall information includes: determining the rainfall level based on the occlusion status information and rainfall information; determining a first wiper setting from a plurality of wiper settings configured in the vehicle based on the rainfall level; and controlling the vehicle's windshield wipers based on the first wiper setting.

[0078] Among them, the rainfall magnitude level is used to indicate the level of rainfall magnitude.

[0079] There are various standards for classifying rainfall intensity, which can be adjusted according to the actual situation. For example, rainfall intensity can be divided into four levels: no rain, light rain, moderate rain, and heavy rain. Or, it can be divided into five levels: no rain, light rain, moderate rain, heavy rain, and torrential rain.

[0080] There are multiple ways to determine the rainfall level based on shading status information and rainfall information, and the specific method can be adjusted according to the actual situation.

[0081] For example, the process of determining the rainfall level based on the occlusion status information and rainfall information is as follows: if the rainfall information is 0, the rainfall level is determined to indicate that the rainfall is 0; if the occlusion status information is occluded and the rainfall information is not 0, the rainfall level is determined to indicate the first rainfall information, which is the minimum rainfall information where the rainfall is not 0; if the occlusion status information is unoccluded and the rainfall information is not 0, the rainfall level is determined to indicate that the rainfall is rainfall information.

[0082] For example, the process of determining the rainfall magnitude level based on occlusion status information and rainfall information is as follows: Determine the target rainfall range to which the rainfall information belongs; there is a one-to-one correspondence between the occlusion status information and the category composed of the rainfall range and the rainfall magnitude level. From multiple rainfall magnitude levels, determine the rainfall magnitude level corresponding to the target rainfall range and the occlusion status information.

[0083] The process of determining the first wiper setting from multiple wiper settings configured in the vehicle based on the rainfall level may include: if the rainfall level indicates a non-zero rainfall amount, then the wiper setting with the lowest frequency is selected from the multiple wiper settings configured in the vehicle as the first wiper setting.

[0084] The first wiper setting indicates the wiper setting used most frequently.

[0085] In some optional embodiments, during the process of controlling the vehicle's windshield wipers according to the first wiper setting, the wiper control method further includes: continuously acquiring new rainfall magnitude levels; if the new rainfall magnitude level is the first rainfall magnitude level, then maintaining the first wiper setting; if the new rainfall magnitude level is the second rainfall magnitude level, then switching the first wiper setting to the second wiper setting, wherein the second wiper setting is higher than the first wiper setting and the second rainfall magnitude level is greater than the first rainfall magnitude level.

[0086] Specifically, continuously acquiring new rainfall intensity levels means continuously acquiring new rainfall information. If the vehicle's occupancy status information remains unchanged, a new rainfall intensity level is determined based on the new rainfall information and the occupancy status information. If the vehicle's occupancy status information changes, a new rainfall intensity level is determined based on the new rainfall information and the changed occupancy status information.

[0087] The first rainfall magnitude level indicates that the new rainfall magnitude level has not changed. The second rainfall magnitude level indicates that the new rainfall magnitude level has changed.

[0088] In some optional embodiments, the rainfall intensity level is configured with an initial wiper setting and an adjustable wiper setting.

[0089] The initial wiper setting indicates the wiper setting directly used to control the vehicle's wipers for that rainfall level. The adjustable wiper setting indicates the wiper setting used to adjust and control the vehicle's wiper switching for that rainfall level.

[0090] Based on this, the second wiper setting mentioned above is the initial wiper setting corresponding to the second rainfall level.

[0091] Based on this, the wiper control method may further include: within a predetermined buffer time after switching to the second wiper setting, counting the target number of times the rainfall level is determined to be the second rainfall level based on the occlusion status information and the detected rainfall information; determining the target probability of the rainfall level being the second rainfall level based on the target number of times; if the target probability is greater than the predetermined probability, switching to the adjustable wiper setting corresponding to the second rainfall level; if the target probability is not greater than the predetermined probability, maintaining the second wiper setting.

[0092] By setting a buffer time and two wiper speeds corresponding to a rainfall level, the risk of incorrect speed shifting can be reduced, and the driver's attention can be diverted due to sudden wiper acceleration, thus ensuring driving safety to a certain extent.

[0093] In some optional embodiments, the above rainfall information is determined based on a multimodal rainfall detection model, and the above shading status information is determined based on a classification model.

[0094] The process of determining the target probability of detecting rainfall of the second rainfall level based on the target number of times may include: determining the initial probability of detecting rainfall of the second rainfall level based on the target number of times; determining the target probability based on the initial probability, a first coefficient, and a second coefficient, wherein the first coefficient is used to indicate the detection accuracy of the multimodal rainfall detection model, and the second coefficient is used to indicate the detection accuracy of the classification model.

[0095] The process of determining the target probability based on the initial probability, the first coefficient, and the second coefficient can be based on regression algorithms, exponential smoothing, least squares, Bayesian filtering, etc.

[0096] Bayesian filtering is preferred for determining the target probability. It is simple and fast, and it is also very effective for multi-classification problems. The complexity will not increase significantly, and the risk of gear shifting errors will be reduced.

[0097] Please see Figure 3 , Figure 3 This is a schematic flowchart of a windshield wiper control method provided in one embodiment of this application. Assume the vehicle's shading status information includes indoor and outdoor conditions. Rainfall intensity levels include no rain (0), light rain (1), moderate rain (2), and heavy rain (3). Rainfall information A includes no rain (0), light rain (1), moderate rain (2), and heavy rain (3). In shading status information B, indoor conditions are marked as 0, and outdoor conditions as A. Wiper speeds include five levels: intermittent wiping, cyclic wiping (level 1), low-speed continuous wiping, cyclic wiping (level 2), and high-speed continuous wiping. Specifically, the initial wiper speed for light rain (level 1) is cyclic wiping (level 1), for moderate rain (level 2) it is low-speed continuous wiping, and for heavy rain (level 3) it is cyclic wiping (level 2). The frequency of different wiper speeds is not fixed and can be adjusted according to vehicle speed and wiper sensitivity.

[0098] Combination Figure 3The following is a specific embodiment to explain the windshield wiper control method of this application. Specifically: The method involves real-time acquisition of images of the vehicle's windshield and sound information from the surrounding environment. The windshield image is input into a background classification model to distinguish whether the vehicle is in an indoor or outdoor environment. The windshield image and sound information are then input into a multimodal rainfall detection model for rainfall detection. Based on the detected rainfall and occlusion status information, the rainfall magnitude, or rainfall level, is determined according to rules. Finally, wiper action commands are output based on the rainfall level and control strategy.

[0099] The rainfall magnitude level is determined by both the rainfall detected by the model and the background environment. The rules include:

[0100] If the multimodal rainfall detection model detects one of the following: light rain, moderate rain, or heavy rain, and the background classification model identifies the vehicle's location as outdoors, then it is determined to be raining, and the rainfall level is the level corresponding to the rainfall information detected by the multimodal rainfall detection model.

[0101] If the multimodal rainfall detection model detects rain, but the background classification model identifies it as indoors, then the rainfall level is uniformly set to light rain (0).

[0102] All other cases were judged as rainless.

[0103] Specifically, the process of determining the rainfall intensity level can be represented by the following formula:

[0104]

[0105] In this context, A represents rainfall information, and B represents the vehicle's shading status information.

[0106] The control strategy may include the following:

[0107] Initially, if the rainfall level is greater than 0 (no rain), the wipers will start automatically and enter intermittent wiping mode.

[0108] If light rain is continuously detected during the intermittent sweeping process, i.e., the rainfall intensity level is light rain 1, then the intermittent sweeping continues.

[0109] During intermittent wiping, if moderate rain is detected (i.e., the rainfall level is moderate rain 2), the wiper will switch to the cycle wiping 1 setting. If the probability P of detecting moderate rain during the buffer time is greater than the threshold, it will directly switch to the low-speed continuous wiping setting; otherwise, it will maintain the cycle wiping 1 setting.

[0110] During intermittent wiping, if heavy rain is detected, i.e., the rainfall level is heavy rain 3, the wiper will switch to the cycle wiping 2 setting. If the probability P of detecting heavy rain during the buffer time is greater than the threshold, it will directly switch to the high-speed continuous wiping setting; otherwise, it will maintain the cycle wiping 2 setting.

[0111] If heavy rain occurs during the low-speed continuous wiping mode, the wiper will switch to the cycle wiping level 2. If the probability P of heavy rain is detected within the buffer time is greater than the threshold, the wiper will be directly controlled to switch from the high-speed continuous wiping mode to the high-speed continuous wiping mode.

[0112] If moderate rain is encountered during high-speed continuous wiping and no heavy rain is detected within the buffer period, then switch back to low-speed continuous wiping mode.

[0113] If light rain is encountered during continuous low-speed wiping and no moderate to heavy rain is detected within the buffer period, the wipers will switch back to the 1st speed setting in the wiper cycle.

[0114] If no rain is detected and there is no rain for the buffer period, the wipers will be turned off.

[0115] The buffer time is set according to the national standard for wiper frequency (45-65 times / minute) and the model recognition accuracy, and is generally no more than 5 seconds.

[0116] The probability P mentioned above is the target probability. The probability calculation process is applied to the continuous determination of rainfall intensity levels. During this buffer period, the wiper speed is adjusted according to vehicle speed and sensitivity, and the wipers enter an adaptive state. Simultaneously, the system statistically analyzes the determined rainfall intensity levels to obtain the proportion of the desired rainfall intensity levels within that time period, and then controls the wiper speed based on this proportion. Assuming the rainfall intensity level corresponding to the switched speed is x, and the detected rainfall state is X, the frequency of X = x during this buffer period is calculated as follows:

[0117]

[0118] Furthermore, since the model recognition results have a certain error, Bayesian filtering can be introduced to reduce the influence of noise. Let the accuracy of the multimodal rainfall detection model A be λ, the accuracy of the background classification model B be μ, and the rainfall state of the first frame image after the buffer time be Z. This can be obtained using Bayesian filtering:

[0119] P(Z=x|A,B)=P(X=X)

[0120] P(Z=x,A,B)=P(A,B)*P(Z=x|A,B)

[0121] Where P(A,B) represents the probability that both the multimodal rainfall detection model A and the background classification model B detect positive examples. Since they are independent, the probability that the final rainfall magnitude level is x is:

[0122] P = λμP(X = x)

[0123] By setting a buffer time when the wipers automatically switch gears, and using Bayesian filtering to calculate the probability, the risk of gear switching errors is reduced. At the same time, the sudden acceleration of the wipers avoids the problem of driver distraction, thus ensuring driving safety to a certain extent.

[0124] To facilitate better implementation of the windshield wiper control method provided in this application, this application also provides an apparatus based on the above-described windshield wiper control method. The meanings of the terms used are the same as in the windshield wiper control method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0125] For example, such as Figure 4 As shown, the windshield wiper control device may include a target image acquisition module 201, a detection information acquisition module 202, a rainfall detection module 203, and a control module 204, as detailed below:

[0126] The target image acquisition module 201 is used to acquire a target image of at least part of the vehicle body;

[0127] The detection information acquisition module 202 is used to acquire detection information collected about the environment in which the vehicle is located;

[0128] Rainfall detection module 203 is used to detect rainfall based on target image and detection information to obtain rainfall information;

[0129] The control module 204 is used to control the vehicle's windshield wipers based on rainfall information.

[0130] In some optional embodiments, the above-mentioned detection information acquisition module 202 includes:

[0131] The detection information acquisition submodule is used to collect sound information about the vehicle's environment to obtain detection information.

[0132] In some optional embodiments, the above-mentioned detection information is rain sound information from the sound information.

[0133] Based on this, the aforementioned detection information acquisition submodule includes:

[0134] The sound acquisition unit is used to collect sound information about the vehicle's surroundings.

[0135] The feature extraction unit is used to extract features from the sound signals contained in the sound information and determine the sound features;

[0136] The rain sound information acquisition unit is used to extract sound signals whose sound features satisfy the rain sound features from the sound information to obtain rain sound information.

[0137] In some optional embodiments, the feature extraction unit described above includes:

[0138] The conversion subunit is used to convert the sound signal contained in the sound information from the linear frequency domain to the Mel frequency domain;

[0139] The cepstral analysis subunit is used to perform cepstral analysis on the sound signal contained in the sound information in the Mel frequency domain to obtain sound features.

[0140] In some optional embodiments, the rainfall detection module 203 described above includes:

[0141] The time alignment submodule is used to determine the target image and rain sound information at the same time based on the capture time of the target image and the acquisition time of the detection information.

[0142] The rainfall detection submodule is used to detect rainfall from target images and rain sound information at the same time based on a multimodal rainfall detection model, and obtain rainfall information.

[0143] In some optional embodiments, the control module 204 further includes:

[0144] The occlusion status information acquisition submodule is used to acquire the occlusion status information of the vehicle. The occlusion status information is used to indicate whether the environment in which the vehicle is located provides shelter from the rain.

[0145] The control submodule is used to control the vehicle's windshield wipers based on shading status information and rainfall information.

[0146] In some optional embodiments, the above-mentioned occlusion status information acquisition submodule includes:

[0147] The classification probability recognition unit is used to identify the classification probability of the environment type in the target image based on the target image using a classification model.

[0148] The occlusion status information determination unit is used to determine the occlusion status information of the vehicle based on the classification probability of the environment type.

[0149] In some optional embodiments, the control submodule described above includes:

[0150] The rainfall intensity level determination unit is used to determine the rainfall intensity level based on the shading status information and rainfall information;

[0151] The first wiper setting determination unit is used to determine the first wiper setting from multiple wiper settings configured in the vehicle based on the amount of rainfall.

[0152] The control unit is used to control the vehicle's windshield wipers according to the first wiper setting.

[0153] In some optional embodiments, the above-mentioned occlusion status information includes occlusion status and unocclusion status.

[0154] Based on this, the above-mentioned rainfall magnitude determination unit includes:

[0155] The first sub-unit for determining rainfall magnitude level is used to determine the rainfall magnitude level indicator as 0 if the rainfall information is 0.

[0156] The second sub-unit for determining rainfall magnitude level is used to determine the first rainfall information as the rainfall magnitude level indicator if the occlusion status information is occlusion status and the rainfall information is not 0. The first rainfall information is the minimum rainfall information where the rainfall magnitude is not 0.

[0157] The third sub-unit for determining rainfall magnitude level is used to determine the rainfall magnitude level indicator as rainfall information if the occlusion status information is unoccluded and the rainfall information is not 0.

[0158] In some optional embodiments, the first wiper mode determination unit described above includes:

[0159] The first wiper setting determination subunit is used to select the wiper setting with the lowest frequency from the multiple wiper settings configured in the vehicle as the first wiper setting if the rainfall indicated by the rainfall level is not 0.

[0160] In some optional embodiments, during the execution of the control unit, the aforementioned wiper control device further includes:

[0161] The continuous data acquisition unit is used to continuously acquire new rainfall magnitude levels;

[0162] The wiper setting remains in the first unit, which is used to maintain the first wiper setting if the new rainfall level is the first rainfall level.

[0163] The first wiper mode switching unit is used to switch the first wiper mode to the second wiper mode if the new rainfall level is the second rainfall level. The second wiper mode is higher than the first wiper mode, and the second rainfall level is higher than the first rainfall level.

[0164] In some optional embodiments, the above-mentioned rainfall level is configured with an initial wiper setting and an adjustable wiper setting, and the second wiper setting is the initial wiper setting corresponding to the second rainfall level.

[0165] Based on this, the aforementioned wiper control device also includes:

[0166] The target count acquisition unit is used to count the number of times the rainfall level is determined to be the second rainfall level based on the occlusion status information and the detected rainfall information within a predetermined buffer time after switching to the second wiper mode.

[0167] The target probability determination unit is used to determine the target probability of the rainfall magnitude level being the second rainfall magnitude level based on the number of target occurrences.

[0168] The second wiper speed control unit is used to switch to the appropriate wiper speed corresponding to the second rainfall level if the target probability is greater than the predetermined probability.

[0169] The second wiper setting is maintained, which is used to maintain the second wiper setting if the target probability is not greater than the predetermined probability.

[0170] In some optional embodiments, the above rainfall information is determined based on a multimodal rainfall detection model, and the above shading status information is determined based on a classification model.

[0171] Based on this, the aforementioned target probability determination unit includes:

[0172] The initial probability determination subunit is used to determine the initial probability of detecting rainfall of the second rainfall level based on the number of target occurrences.

[0173] The target probability determination subunit is used to determine the target probability based on the initial probability, a first coefficient, and a second coefficient. The first coefficient is used to indicate the detection accuracy of the multimodal rainfall detection model, and the second coefficient is used to indicate the detection accuracy of the classification model.

[0174] Therefore, in the windshield wiper control device provided in this application embodiment, the target image acquisition module 201 acquires a target image of at least a portion of the vehicle body; the detection information acquisition module 202 acquires detection information collected regarding the vehicle's environment; the rainfall detection module 203 performs rainfall detection based on the target image and the detection information to obtain rainfall information; and the control module 204 controls the vehicle's windshield wipers based on the rainfall information. Thus, by combining the image of at least a portion of the vehicle body with the rainfall information obtained from rainfall detection based on the detection information collected regarding the vehicle's environment, control of the windshield wipers is achieved, thereby improving the accuracy of controlling the windshield wipers during operation.

[0175] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0176] This application also provides an electronic device, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0177] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0178] The processor 301 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes computer programs and / or modules stored in the memory 302, and calls data stored in the memory 302 to perform various functions and process data. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0179] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and performs security controls by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as sound playback function, navigation function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0180] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0181] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0182] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 302 according to the following instructions, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions, such as:

[0183] Acquire target images of at least a portion of the vehicle body;

[0184] Acquire detection information about the vehicle's environment;

[0185] Rainfall information is obtained by detecting rainfall based on target images and detection information.

[0186] Control the vehicle's windshield wipers based on rainfall information.

[0187] Therefore, the electronic device provided in this application acquires a target image of at least a portion of the vehicle body; acquires detection information collected about the vehicle's environment; performs rainfall detection based on the target image and the detection information to obtain rainfall information; and controls the vehicle's windshield wipers based on the rainfall information. Thus, by combining the image of at least a portion of the vehicle body with the rainfall information obtained from rainfall detection based on the detection information collected about the vehicle's environment, control of the windshield wipers is achieved, thereby improving the accuracy of windshield wiper operation.

[0188] For details on the specific implementation methods and corresponding beneficial effects of each of the above operations, please refer to the detailed description of the wiper control method above, which will not be repeated here.

[0189] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a storage medium and loaded and executed by a processor.

[0190] Therefore, embodiments of this application provide a storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the windshield wiper control methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0191] Acquire target images of at least a portion of the vehicle body;

[0192] Acquire detection information about the vehicle's environment;

[0193] Rainfall information is obtained by detecting rainfall based on target images and detection information.

[0194] Control the vehicle's windshield wipers based on rainfall information.

[0195] Therefore, the storage medium provided in this application acquires target images of at least a portion of the vehicle body; acquires detection information collected about the vehicle's environment; performs rainfall detection based on the target images and detection information to obtain rainfall information; and controls the vehicle's windshield wipers based on the rainfall information. Thus, by combining images of at least a portion of the vehicle body with rainfall information obtained from rainfall detection based on detection information collected about the vehicle's environment, control of the windshield wipers is achieved, thereby improving the accuracy of windshield wiper operation.

[0196] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0197] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0198] Since the computer program stored in the storage medium can execute the steps of any of the wiper control methods provided in the embodiments of this application, the beneficial effects that any of the wiper control methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0199] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the aforementioned windshield wiper control method.

[0200] This application also provides a vehicle that includes the aforementioned electronic device. The electronic device can be installed on the vehicle or integrated into a device on the vehicle for controlling the device. The specific structure of the vehicle is not limited in this application. The specific implementation methods and corresponding beneficial effects of the various operations of the electronic device described above are also applicable to this vehicle. For details, please refer to the detailed description of the vehicle rear-collision safety control method above, which will not be repeated here.

[0201] The foregoing has provided a detailed description of a windshield wiper control method, device, electronic device, storage medium, computer program product, and vehicle provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A wiper control method characterized by, The method comprises: acquiring a target image photographed for at least part of a vehicle body of a vehicle; acquiring detection information collected for an environment in which the vehicle is located; performing rain amount detection according to the target image and the detection information to obtain rain amount information; controlling a rain wiper of the vehicle according to the rain amount information.

2. The wiper control method according to claim 1, characterized by, The acquiring of the detection information collected for the environment in which the vehicle is located comprises: collecting sound information of the environment in which the vehicle is located to obtain the detection information.

3. The wiper control method according to claim 2, characterized by, The detection information is rain sound information in the sound information; the collecting of the sound information of the environment in which the vehicle is located to obtain the detection information comprises: collecting sound information of the environment in which the vehicle is located; extracting a sound feature of a sound signal contained in the sound information to determine the sound feature; extracting a sound signal whose sound feature satisfies a rain sound feature from the sound information to obtain the rain sound information.

4. The wiper control method according to claim 3, characterized by, The extracting of the sound feature of the sound signal contained in the sound information to determine the sound feature comprises: converting the sound signal contained in the sound information from a linear frequency domain to a mel frequency domain; performing a cepstrum analysis on the sound signal contained in the sound information in the mel frequency domain to obtain the sound feature.

5. The rain-repellent control method according to claim 3, characterized by, The performing of the rain amount detection according to the target image and the detection information to obtain the rain amount information comprises: determining the target image and the rain sound information at the same time according to a photographing time of the target image and a collection time of the detection information; performing rain amount detection on the target image and the rain sound information at the same time according to a multi-modal rain amount detection model to obtain the rain amount information.

6. The method of claim 1 to 5, wherein The controlling of the rain wiper of the vehicle according to the rain amount information further comprises: acquiring shielding state information of the vehicle, the shielding state information being used to indicate whether the environment in which the vehicle is located is shielding rain for the vehicle; controlling the rain wiper of the vehicle according to the shielding state information and the rain amount information.

7. The wiper control method according to claim 6, characterized by, The acquiring of the shielding state information of the vehicle comprises: recognizing, by a classification model, a classification probability of an environment type of the environment in which the vehicle is located in the target image based on the target image; determining the shielding state information of the vehicle according to the classification probability of the environment type.

8. The wiper control method according to claim 6, characterized by, The controlling of the rain wiper of the vehicle according to the shielding state information and the rain amount information comprises: determining a rain amount size level according to the shielding state information and the rain amount information; determining a first rain wiper gear from a plurality of rain wiper gears configured for the vehicle according to the rain amount size level; controlling the rain wiper of the vehicle according to the first rain wiper gear.

9. The wiper control method according to claim 8, characterized by, The shielding state information comprises a shielding state and a non-shielding state; The determining of the rain amount size level according to the shielding state information and the rain amount information comprises: if the rain amount information is 0, determining that the rain amount size level indicates that a rain amount size is 0; if the shielding state information is the shielding state and the rain amount information is not 0, determining that the rain amount size level indicates first rain amount information, the first rain amount information being minimum rain amount information whose rain amount size is not 0. If the blocking state information is the unblocked state and the rainfall information is not 0, it is determined that the rainfall size level indicates the rainfall size as the rainfall information.

10. The rain-repellent control method according to claim 8, characterized by, The first wiper gear is determined from a plurality of wiper gears configured for the vehicle according to the rainfall size level. If the rainfall size indicated by the rainfall size level is not 0, the wiper gear with the minimum frequency is selected from the plurality of wiper gears configured for the vehicle as the first wiper gear.

11. The wiper control method according to claim 10, characterized by, In the process of controlling the wiper of the vehicle according to the first wiper gear, the method further comprises: continuously obtaining a new rainfall size level; if the new rainfall size level is the first rainfall size level, the first wiper gear is maintained; if the new rainfall size level is the second rainfall size level, the first wiper gear is switched to the second wiper gear, the second wiper gear is higher than the first wiper gear, and the second rainfall size level is greater than the first rainfall size level.

12. The wiper control method according to claim 11, characterized by, The rainfall size level corresponds to an initial wiper gear and an adjusted wiper gear, and the second wiper gear is the initial wiper gear corresponding to the second rainfall size level. The method further comprises: within a predetermined buffer time after switching to the second wiper gear, a target number of times that the rainfall size level determined based on the blocking state information and the detected rainfall information is the second rainfall size level is counted; based on the target number of times, a target probability that the rainfall size level is the second rainfall size level is determined; if the target probability is greater than a predetermined probability, the second rainfall size level is switched to the adjusted wiper gear corresponding to the second rainfall size level; if the target probability is not greater than the predetermined probability, the second wiper gear is maintained.

13. The wiper control method according to claim 12, characterized by, The rainfall information is determined based on a multi-modal rainfall detection model, and the blocking state information is determined based on a classification model. Based on the target number of times, the target probability that the rainfall size level is the second rainfall size level is determined, comprising: based on the target number of times, an initial probability that the rainfall size level is the second rainfall size level is determined; based on the initial probability, a first coefficient and a second coefficient, a target probability is determined, the first coefficient is used to indicate the detection accuracy of the multi-modal rainfall detection model, and the second coefficient is used to indicate the detection accuracy of the classification model.

14. A wiper control device characterized by comprising: The device comprises: a target image acquisition module configured to acquire a target image taken at least partially on a vehicle body of a vehicle; a detection information acquisition module configured to acquire detection information collected for an environment in which the vehicle is located; a rainfall detection module configured to perform rainfall detection based on the target image and the detection information to obtain rainfall information; a control module configured to control a wiper of the vehicle based on the rainfall information.

15. An electronic device, comprising: The device comprises one or more processors and a memory storing a computer program, when the computer program is executed by the processor, the processor executes the steps of the wiper control method in any one of claims 1 to 13. The device comprises one or more processors and a memory storing a computer program, when the computer program is executed by the processor, the processor executes the steps of the wiper control method in any one of claims 1 to 13.

16. A storage medium, characterized by A computer program including instructions for causing a processor to perform the steps of the wiper control method of any one of claims 1 to 13 when the computer program is run on the processor.

17. A computer program product, characterised in that, A computer program or instructions for causing a processor to perform the steps of the wiper control method of any one of claims 1 to 13 when the computer program or instructions are executed by the processor.

18. A vehicle characterized by comprising: The vehicle includes the electronic device of claim 15.

Images