Rainfall identification method, electronic equipment and vehicle

Adjusting the rainfall recognition model through real-time video data and reinforcement learning algorithms, the problem of low recognition accuracy of single images is solved, and accurate rainfall recognition and automatic adjustment of wipers under dynamic changes in rainfall are achieved to ensure safe vehicle traffic.

CN119027896APending Publication Date: 2024-11-26GREAT WALL MOTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310601360.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the prior art, relying on a single image for rainfall recognition will affect the accuracy of the recognition, especially when the rainfall changes dynamically, resulting in inaccurate adjustment of wiper wiper wiper wiper wiper wiper wiper wiper, which may lead to incorrect startup or inability to remove rainwater from the vehicle, affecting Vehicles pass.

Method used

By obtaining real-time video data over a period of time, using the pre-trained rainfall recognition model for rainfall recognition, and adjusting the model in combination with reinforcement learning algorithms to correspond to scenes of dynamic changes in rainfall in real time to improve the accuracy of rainfall recognition.

Benefits of technology

Accurate rainfall recognition under dynamic changes in rainfall is achieved, which avoids misstarting start and vehicle traffic problems caused by single image recognition, improves the automatic adjustment accuracy of wiper wiper wiper strength, and ensures effective removal of rainwater on the vehicle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027896A_ABST
    Figure CN119027896A_ABST
Patent Text Reader

Abstract

The invention provides a rainfall identification method, electronic equipment and a vehicle, and the method comprises the steps: obtaining to-be-predicted weather data, enabling the to-be-predicted weather data to comprise a real-time video in a period of time, inputting the real-time video into a rainfall identification model which is trained in advance, and obtaining a rainfall prediction result of the to-be-predicted weather data, through the real-time video, the scene corresponding to the rainfall dynamic change in real time, the rainfall recognition is carried out based on the real-time video by using the rainfall recognition model which is trained in advance, the accuracy of rainfall recognition can be ensured, and the problem that the recognition accuracy is influenced by rainfall recognition depending on a single image is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a rainfall recognition method, electronic equipment and vehicle. Background Art

[0002] In order to ensure safe driving, the wiping intensity of the wipers can be automatically adjusted according to the rainfall. An AI (Artificial Intelligence) model is usually used to realize the recognition of rainfall.

[0003] However, related technologies often use AI models to identify rainfall based on a single image. Since rainfall changes dynamically, relying on a single image to identify rainfall will affect the accuracy of recognition. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose a rainfall recognition method, electronic equipment and vehicle to solve the problem in the prior art that rainfall recognition based on a single image will affect the recognition accuracy.

[0005] Based on the above purpose, the first aspect of the present application provides a rainfall identification method, comprising:

[0006] Acquire weather data to be predicted, where the weather data to be predicted includes real-time video within a period of time;

[0007] The real-time video is input into a pre-trained rainfall recognition model to obtain a rainfall prediction result of the weather data to be predicted.

[0008] Optionally, the rainfall recognition model is obtained through the following training process, including:

[0009] Acquire a real-time weather data sample and a rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and determine a reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, wherein the reward value is negatively correlated with a first difference, and the first difference is a difference between the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and an actual rainfall level;

[0010] Inputting the real-time weather data sample into a pre-trained initial rainfall recognition model to obtain a rainfall prediction result corresponding to the real-time weather data sample;

[0011] Based on the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and the reward values ​​corresponding to the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, a loss function is constructed using a reinforcement learning algorithm, and the initial rainfall recognition model is adjusted using the loss function to obtain a rainfall recognition model.

[0012] Optionally, the initial rainfall recognition model includes a first initial rainfall recognition model and a second initial rainfall recognition model, the real-time weather data sample is subjected to the first initial rainfall recognition model to output a first rainfall prediction result, and the real-time weather data sample is subjected to the second initial rainfall recognition model to output a second rainfall prediction result;

[0013] The method comprises: constructing a loss function based on the real-time weather data sample, the rainfall prediction result corresponding to the real-time weather data sample, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle by using a reinforcement learning algorithm, and adjusting the initial rainfall recognition model by using the loss function to obtain a rainfall recognition model, including:

[0014] Constructing a first loss function using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle;

[0015] Based on the second rainfall prediction result and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, constructing a second loss function using a reinforcement learning algorithm;

[0016] Adjusting the first initial rainfall recognition model by using the first loss function to obtain an adjusted first initial rainfall recognition model;

[0017] The second initial rainfall recognition model is adjusted by using the second loss function to obtain an adjusted second initial rainfall recognition model, and the adjusted first initial rainfall recognition model and the adjusted second initial rainfall recognition model are used as the rainfall recognition model.

[0018] Optionally, constructing the first loss function using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle includes:

[0019] Performing a ratio calculation based on the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value;

[0020] A second intermediate value is obtained by performing a calculation based on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result;

[0021] Performing a weighted sum calculation based on multiple second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value;

[0022] The first loss function is constructed based on the first intermediate value and the third intermediate value.

[0023] Optionally, the calculating a ratio of the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value includes:

[0024] Determine, as a first probability, a probability that a first rainfall prediction result output by the first initial rainfall recognition model of the real-time weather data sample belongs to a target rainfall level;

[0025] Determining, as a second probability, a probability that a rainfall prediction result corresponding to the real-time weather data sample on the vehicle belongs to the target rainfall level;

[0026] A ratio calculation is performed using the first probability and the second probability to obtain a ratio result, and the ratio result is used as the first intermediate value.

[0027] Optionally, the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result are processed to obtain a second intermediate value, including:

[0028] Acquire the time series information corresponding to the real-time weather data sample, and determine, based on the time series information, a next real-time weather data sample corresponding to the next time series information relative to the time series information;

[0029] A summation is performed using the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result corresponding to the next real-time weather data sample to obtain a summation result;

[0030] A difference calculation is performed on the sum result and the second rainfall prediction result corresponding to the real-time weather data sample to obtain a first difference result, and the first difference result is used as the second intermediate value.

[0031] Optionally, performing weighted sum calculation on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value includes:

[0032] Performing weight matching on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to determine a plurality of weights corresponding to the plurality of second intermediate values;

[0033] According to the multiple weights, weighted summation is performed on the multiple second intermediate values ​​to obtain a weighted summation result, and the weighted summation result is used as the third intermediate value.

[0034] Optionally, constructing the second loss function using a reinforcement learning algorithm based on the second rainfall prediction result and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle includes:

[0035] Performing a decaying cumulative calculation on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle to obtain a decaying cumulative result;

[0036] performing a difference calculation on the second rainfall prediction result and the attenuation accumulation result to obtain a second difference result;

[0037] The second loss function is constructed according to the second difference result.

[0038] Based on the same inventive concept, the second aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the method described in the first aspect is implemented when the processor executes the program.

[0039] Based on the same inventive concept, the third aspect of the present application provides a vehicle, comprising the electronic device described in the second aspect.

[0040] From the above, it can be seen that the rainfall recognition method, electronic device and vehicle provided by the present application obtain weather data to be predicted, which includes real-time video within a period of time, and input the real-time video into a pre-trained rainfall recognition model to obtain the rainfall prediction result of the weather data to be predicted. The real-time video corresponds to the scene of dynamic changes in rainfall in real time, so that the rainfall recognition model that has been pre-trained can be used to perform rainfall recognition based on real-time video, which can ensure the accuracy of rainfall recognition and avoid the problem that relying on a single image for rainfall recognition will affect the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A flowchart of a rainfall identification method according to an embodiment of the present application;

[0043] Figure 2A A schematic diagram of capturing weather video data according to an embodiment of the present application;

[0044] Figure 2B This is a schematic diagram of the structure of the initial rainfall recognition model in an embodiment of the present application;

[0045] Figure 2C A schematic diagram of the rainfall recognition model training of an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a rainfall identification device according to an embodiment of the present application;

[0047] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0049] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0050] In related technologies, AI models are often used to identify rainfall based on a single image. Since rainfall changes dynamically, relying on a single image to identify rainfall will affect the accuracy of recognition.

[0051] In addition, the amount of rainfall can be identified through a rain sensor. By taking advantage of the different light refractive indices of water, air and glass and the strong absorption capacity of water in the infrared band, the amount of rainfall can be identified and judged based on the change in reflectivity. However, it is prone to false start-up and wrong judgment, and the sensing area is small.

[0052] For the recognition of rainfall, data can also be collected through the DVR (Digital Video Recorder) camera on the vehicle, and the AI ​​model can be used to identify the amount of rainfall and control the wipers according to the amount of rainfall to replace the function of the rain sensor. However, the training method of the AI ​​model used: data collection - data labeling - model training, has many problems.

[0053] For example, offline data collection is difficult to cover many scenarios, the cycle is long, and the technology requires data on specific weather conditions. Optimizing an AI model requires a large amount of comprehensive data, and it is difficult to optimize the model to a high degree relying solely on limited data collected offline.

[0054] Secondly, there is the problem of data labeling. The data is about dynamic rain conditions, and the rain conditions before and after the wipers will also change greatly. Judging the size of the wipers is a time series process. In addition, it is difficult to control the labeling and judgment criteria for a single piece of data, which leads to difficulties in labeling and poor data quality. Naturally, various problems will occur in the trained model.

[0055] A rainfall recognition method proposed in this embodiment is applied to a terminal device, which can be a server or a vehicle terminal. Through real-time video, the scene of dynamic change of rainfall can be corresponded in real time, so that the rainfall recognition model that has been trained in advance can be used to perform rainfall recognition based on real-time video, which can ensure the accuracy of rainfall recognition and avoid the problem that rainfall recognition based on a single image will affect the recognition accuracy. Among them, the rainfall recognition method of the present application can be applied in fields such as meteorology, hydrology and flood control, disaster warning or vehicle fields. Here, the vehicle field is preferred. In the vehicle field, the DVR camera on the vehicle is used to collect weather data to be predicted in real time. The weather data to be predicted includes real-time video within a period of time. The collected real-time video is input into a pre-trained rainfall recognition model, and the rainfall recognition model outputs a rainfall prediction result corresponding to the weather data to be predicted, such as Figure 1 As shown, including:

[0056] Step 101: Obtain weather data to be predicted, where the weather data to be predicted includes real-time video within a period of time.

[0057] In this step, the weather data to be predicted is the video data collected online in real time by the vehicle's own DVR camera. The real-time collected weather data to be predicted is divided into multiple real-time videos. For example, according to the time sequence of collecting the weather data to be predicted, every two seconds (which can be set according to the specific situation) is divided into a real-time video. As a weather data to be predicted, the real-time video can correspond to the scene of dynamic changes in rainfall in real time.

[0058] Step 102: input the real-time video into a pre-trained rainfall recognition model to obtain a rainfall prediction result of the weather data to be predicted.

[0059] In this step, each real-time video is input into a pre-trained rainfall recognition model in sequence according to the time sequence of acquisition, and the rainfall recognition model outputs the rainfall prediction results corresponding to each real-time video segment in sequence. That is, for each real-time video divided from the weather data to be predicted, the rainfall recognition model will output the rainfall prediction results corresponding to each real-time video, so that it can respond in real time to different rainfall intensities caused by dynamic changes in rainfall, so as to automatically adjust the wiping intensity of the wipers according to the rainfall prediction results corresponding to each real-time video.

[0060] The rainfall prediction result indicates the rainfall intensity level, for example, no rain, weak, medium, strong, and extremely strong. The rainfall prediction result may also be a sequence number, which is used to indicate each rainfall intensity level. For example, the rainfall prediction result is 1, which indicates that the rainfall intensity level is weak. Then, the wiper intensity of the vehicle can be automatically adjusted according to the weak rainfall intensity level.

[0061] Through the above scheme, by obtaining the weather data to be predicted, the weather data to be predicted includes real-time video within a period of time, and inputting the real-time video into a pre-trained rainfall recognition model, the rainfall prediction result of the weather data to be predicted is obtained. The real-time video corresponds to the scene of dynamic change of rainfall in real time, so that the rainfall recognition model that has been pre-trained can be used to perform rainfall recognition based on real-time video, which can ensure the accuracy of rainfall recognition and avoid the problem that relying on a single image for rainfall recognition will affect the recognition accuracy.

[0062] In addition, it can also avoid the problem of excessive power consumption caused by incorrect identification or the inability to clear rain water on the vehicle affecting the vehicle's passage.

[0063] In some embodiments, the rainfall recognition model is obtained through the following training process, including:

[0064] Step A1, obtain the real-time weather data sample and the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and determine the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, wherein the relationship between the reward value and the first difference is negatively correlated, and the first difference is the difference between the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the actual rainfall level.

[0065] In this step, the real-time weather data samples are collected online in real time using the DVR camera on the vehicle, which can cover various weather scenes in real time, so the corresponding rainfall recognition results are more real-time and comprehensive. For example, the DVR camera on the vehicle collects real-time weather data samples in real time, and divides every two seconds into a video segment according to the time sequence of collection, as a real-time weather data sample.

[0066] Obtain the rainfall forecast result corresponding to the real-time weather data sample on the vehicle, and calculate the reward value based on the actual rainfall level reported by the sensor or user. The smaller the difference between the two (i.e., the first difference), the greater the reward value. The reward value can be expressed as:

[0067] reward=max_class-abs(gt_class-pred_class)

[0068] Among them, max_class is the maximum value of the rainfall intensity level, gt_class is the actual rainfall level, pred_class is the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and reward is the reward value.

[0069] Step A2: input the real-time weather data sample into a pre-trained initial rainfall recognition model to obtain a rainfall prediction result corresponding to the real-time weather data sample.

[0070] In this step, the rainfall prediction result indicates the rainfall intensity level, for example, no rain, weak, moderate, strong, and extremely strong. The rainfall prediction result may also be a sequence number, which is used to indicate each rainfall intensity level. For example, the rainfall prediction result is 1, which indicates that the rainfall intensity level is weak. The initial rainfall recognition model is a model that has been trained in advance.

[0071] For example, the DVR camera on the vehicle collects real-time weather data samples in real time. According to the time sequence of collection, every two seconds is divided into a video segment. As a real-time weather data sample, each video segment is input into the pre-trained initial rainfall recognition model in sequence according to the time sequence of collection. The initial rainfall recognition model classifies and identifies the rainfall intensity level of each video segment and outputs the corresponding rainfall prediction result, thus completing the recognition of the rainfall intensity level of the real-time weather data sample.

[0072] Step A3, based on the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and the reward values ​​corresponding to the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, a loss function is constructed using a reinforcement learning algorithm, and the initial rainfall recognition model is adjusted using the loss function to obtain a rainfall recognition model.

[0073] In this step, for example, the DVR camera on the vehicle collects real-time weather data samples in real time, and divides each video segment into a video segment every two seconds according to the time sequence of collection. As a real-time weather data sample, each video segment is sequentially input into the pre-trained initial rainfall recognition model according to the time sequence of collection. The initial rainfall recognition model classifies and recognizes which rainfall intensity level each video segment belongs to, and then uses the reinforcement learning algorithm (Proximal Policy Optimization, PPO) to construct a loss function based on each video segment and the rainfall prediction result corresponding to each video segment. The online real-time weather data sample can cover various weather scenes in real time, so the corresponding rainfall recognition result is also more real-time and comprehensive, which can ensure the effect of the constructed loss function on the trained initial rainfall recognition model. In addition, the loss function constructed by the reinforcement learning algorithm requires less data to quickly complete the adjustment process for the adjustment of the loss function, avoiding the time-consuming and expensive problems of collecting data.

[0074] The initial rainfall recognition model is adjusted through the loss function, and the parameters of the initial rainfall recognition model are adjusted by minimizing the loss function to obtain the rainfall recognition model, thereby completing the continued optimization process based on the initial rainfall recognition model, thereby improving the training effect of the rainfall recognition model and ensuring the accuracy of the rainfall recognition model for rainfall recognition.

[0075] Through the above scheme, by obtaining online real-time weather data samples instead of historical weather data collected offline, the problem of incomplete weather scenes covered by offline historical weather data can be avoided. The real-time weather data samples are input into the pre-trained initial rainfall recognition model, and the initial rainfall recognition model outputs the rainfall recognition results corresponding to the real-time weather data samples. Then, the loss function is constructed through the reinforcement learning algorithm. Since the online real-time weather data samples can cover various weather scenes in real time, the corresponding rainfall recognition results are also more real-time and comprehensive, which can ensure that the constructed loss function can adjust the trained initial rainfall recognition model again to obtain the rainfall recognition model, thereby completing the process of continued optimization based on the initial rainfall recognition model, thereby improving the training effect of the rainfall recognition model and ensuring the accuracy of the rainfall recognition model for rainfall recognition.

[0076] In addition, online real-time weather data samples can cover more comprehensive weather scenarios than offline historical weather data, and the loss function constructed using real-time weather data samples and corresponding rainfall forecast results is used to adjust the trained initial rainfall recognition model again. This can also avoid the problem that relying solely on offline historical weather data may affect the training process of the rainfall recognition model due to incomplete coverage of weather scenarios, differences in vehicle models, window tilt angles, window bend angles, window thicknesses, etc.

[0077] In some embodiments, the initial rainfall recognition model includes a first initial rainfall recognition model and a second initial rainfall recognition model. The real-time weather data sample outputs a first rainfall prediction result after passing through the first initial rainfall recognition model, and the real-time weather data sample outputs a second rainfall prediction result after passing through the second initial rainfall recognition model.

[0078] Step A3 includes:

[0079] Step A31, constructing a first loss function using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle.

[0080] Step A32, constructing a second loss function using a reinforcement learning algorithm based on the reward value corresponding to the second rainfall prediction result and the rainfall prediction result corresponding to the real-time weather data sample on the vehicle.

[0081] Step A33: adjusting the first initial rainfall recognition model by using the first loss function to obtain an adjusted first initial rainfall recognition model.

[0082] Step A34, using the second loss function to adjust the second initial rainfall recognition model to obtain an adjusted second initial rainfall recognition model, and using the adjusted first initial rainfall recognition model and the adjusted second initial rainfall recognition model as the rainfall recognition model.

[0083] In the above solution, the initial rainfall recognition model stored in the terminal device includes a first initial rainfall recognition model and a second initial rainfall recognition model.

[0084] The linear layer used to output the classification results in the structures of the first initial rainfall recognition model and the second initial rainfall recognition model are different, and the other layers are the same. Among them, the linear layer used to output the rainfall prediction results in the first initial rainfall recognition model is MLPHead (fully connected neural network), and the linear layer used to output the rainfall prediction results in the second initial rainfall recognition model is fc (fully connected layer).

[0085] The first loss function is constructed using a reinforcement learning algorithm using the first rainfall prediction result outputted from the first initial rainfall recognition model inputted by the real-time weather data sample obtained in the data collection phase, the second rainfall prediction model outputted from the second initial rainfall recognition model inputted by the real-time weather data sample, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle. Then, the parameters of the first initial rainfall recognition model are adjusted by minimizing the first loss function to obtain an adjusted first initial rainfall recognition model.

[0086] Similarly, the second rainfall prediction result outputted from the second initial rainfall recognition model is inputted using the real-time weather data sample, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle is used to construct the second loss function using the reinforcement learning algorithm. Then, the parameters of the second initial rainfall recognition model are adjusted by minimizing the second loss function to obtain an adjusted second initial rainfall recognition model.

[0087] The adjusted first initial rainfall recognition model and the adjusted second initial rainfall recognition model are used as the rainfall recognition model, updated and arranged on the vehicle for rainfall recognition.

[0088] In some embodiments, step A31 includes:

[0089] Step A311, performing ratio calculation based on the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value.

[0090] Step A312, performing calculations based on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result to obtain a second intermediate value.

[0091] Step A313, performing weighted sum calculation on multiple second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value.

[0092] Step A314: construct the first loss function based on the first intermediate value and the third intermediate value.

[0093] In the above scheme, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle (a model with the same structure as the first initial rainfall recognition model before being adjusted by the first loss function) is combined with the actual rainfall level fed back by the sensor or the user to calculate the reward value. The smaller the difference between the two, the greater the reward value. The calculated reward value can be expressed as:

[0094] reward=max_class-abs(gt_class-pred_class)

[0095] Among them, max_class is the maximum value of the rainfall intensity level, gt_class is the actual rainfall level, pred_class is the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and reward is the reward value.

[0096] The terminal device receives and stores real-time weather data samples, rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and reward values, and constructs a first loss function based on the first rainfall prediction result and the second rainfall prediction result to adjust the first initial rainfall recognition model. After adjustment, the first initial rainfall recognition model and the adjusted second initial rainfall recognition model are updated on the vehicle together.

[0097] Among them, the first loss function can be expressed as:

[0098]

[0099] Among them, L CLIP (θ) represents the first loss function, CLIP represents the real-time weather data sample, θ represents the parameters of the first initial rainfall recognition model, represents the expected operation, t represents the collection order of real-time weather data samples, r t (θ) represents the first intermediate value, Represents the third intermediate value, clip represents the truncation operation, which means dividing the first intermediate value less than 1-∈ into 1-∈, and dividing the first intermediate value greater than 1+∈ into 1+∈, ∈ represents the hyperparameter, and min represents the minimum value.

[0100] The parameters of the first initial rainfall identification model are adjusted by minimizing the first loss function.

[0101] In some embodiments, step A311 includes:

[0102] Step A3111, determining the probability that the first rainfall prediction result output by the first initial rainfall recognition model of the real-time weather data sample belongs to the target rainfall level as the first probability.

[0103] Step A3112, determining the probability that the rainfall prediction result corresponding to the real-time weather data sample on the vehicle belongs to the target rainfall level as the second probability.

[0104] Step A3113, using the first probability and the second probability to perform a ratio calculation to obtain a ratio result, and using the ratio result as the first intermediate value.

[0105] In the above scheme, for example, the video segments (i.e., real-time weather data samples) sorted as t are input into the first initial rainfall recognition model according to the acquisition time sequence, and the output first rainfall prediction result belongs to a t The probability of is the first probability.

[0106] The rainfall prediction result output by the model on the vehicle for the video segment ranked t (i.e., real-time weather data sample) belongs to a t The probability of is taken as the second probability.

[0107] The first probability is compared with the second probability to obtain a first intermediate value, wherein the first intermediate value can be expressed as:

[0108]

[0109] Among them, r t (θ) represents the first intermediate value, a t represents the target rainfall level, t represents the collection order of real-time weather data samples, s t represents the real-time weather data sample sorted as t, π θ (a t |s t ) represents the first probability, represents the second probability.

[0110] In some embodiments, step A312 includes:

[0111] Step A3121, obtaining the timing information corresponding to the real-time weather data sample, and determining, based on the timing information, the next real-time weather data sample corresponding to the next timing information relative to the timing information.

[0112] Step A3122, using the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result corresponding to the next real-time weather data sample to perform sum calculation to obtain the sum result.

[0113] Step A3123, performing a difference calculation on the summed result and the second rainfall prediction result corresponding to the real-time weather data sample to obtain a first difference result, and using the first difference result as the second intermediate value.

[0114] In the above scheme, the timing information represents the order of collection of real-time weather data samples, and is used to sort the real-time weather data samples. For example, if the timing information corresponding to the real-time weather data sample is t, then the next timing information corresponding to the timing information t is t+1. According to the next timing information t+1, the real-time weather data sample sorted as t+1 is determined as the next real-time weather data sample.

[0115] The reward value calculated by combining the first rainfall prediction result with the actual rainfall level fed back by the sensor or the user is added together with the second rainfall prediction result output by the second initial rainfall recognition model of the real-time weather data sample ranked as t+1 to obtain the added result.

[0116] Then, the addition result and the second rainfall prediction result output by the second initial rainfall recognition model are used to perform a difference calculation to obtain a difference result, which is the second intermediate value.

[0117] The second intermediate value can be expressed as:

[0118] δ t =r t +γV(s t+1 )-V(s t )

[0119] Among them, δ t It is represented as the second middle value corresponding to the real-time weather data sample ranked as t, r t represents the reward value, γ represents the decay coefficient of the reward value over time, s t represents the real-time weather data sample sorted as t, V(s t ) represents the second rainfall prediction result output by the second initial rainfall recognition model for the real-time weather data sample ranked as t, s t+1 represents the real-time weather data sample sorted as t+1, V(s t+1 ) represents the second rainfall prediction result output by the second initial rainfall recognition model of the real-time weather data sample ranked as t+1.

[0120] In some embodiments, step A313 includes:

[0121] Step A3131, weight matching is performed on multiple second intermediate values ​​corresponding to the real-time weather data samples to determine multiple weights corresponding to the multiple second intermediate values.

[0122] Step A3132: Perform weighted summation on the multiple second intermediate values ​​according to the multiple weights to obtain a weighted summation result, and use the weighted summation result as the third intermediate value.

[0123] In the above scheme, for example, the real-time weather data samples are video segments ranked from t to T-1 (ie, real-time weather data samples), and each video segment corresponds to a second intermediate value, which is a matching weight for the second intermediate value corresponding to each video segment.

[0124] The second intermediate values ​​corresponding to the video segments ranked from t to T-1 are weightedly summed according to the matching weights to obtain a weighted sum result, which is the third intermediate value.

[0125] Among them, the third intermediate value can be expressed as:

[0126]

[0127] in, represents the third intermediate value, δ t It is represented by the second middle value corresponding to the real-time weather data sample ranked as t, δ t+1 represents the second intermediate value corresponding to the real-time weather data sample ranked as t+1, δ T-1 represents the second middle value corresponding to the real-time weather data sample ranked as T-1, γλ represents δ t+1 The weight of , in γλ, λ represents δ t+1 The decay coefficient of the reward value over time, γ represents the decay coefficient of the reward value over time, γλ T-t+1 Denotes δ T-1 The weight of T-t+1 λ represents δ T-1 The decay coefficient of the reward value over time, γ represents the decay coefficient of the reward value over time, T represents the ranking of the real-time weather data samples is T, and t represents the ranking of the real-time weather data samples is t.

[0128] In some embodiments, step A32 includes:

[0129] Step A321, performing a decay accumulation calculation on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle to obtain a decay accumulation result.

[0130] Step A322, performing a difference calculation on the second rainfall prediction result and the attenuation accumulation result to obtain a second difference result.

[0131] Step A323, performing a difference calculation on the second rainfall prediction result and the attenuation accumulation result to obtain a second difference result.

[0132] In the above scheme, the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle is subjected to attenuation accumulation calculation to obtain the attenuation accumulation result, which can be expressed as follows:

[0133]

[0134] in, Represents the attenuation accumulation result, r t represents the reward value, and γ represents the decay coefficient of the reward value over time.

[0135] The second rainfall prediction result outputted from the second initial rainfall recognition model is inputted using real-time weather data samples, and a difference calculation is performed in combination with the attenuation accumulation result to obtain a second difference result, which is then squared to construct a second loss function. The parameters of the second initial rainfall recognition model are adjusted by minimizing the second loss function, and after adjustment, the second initial rainfall recognition model and the adjusted first initial rainfall recognition model are updated and loaded onto the vehicle together.

[0136] Among them, the second loss function can be expressed as:

[0137]

[0138] in, represents the second loss function, Represents the attenuation accumulation result, s t represents the real-time weather data sample with the order t, t represents the real-time weather data sample with the order t, V θ (s t ) represents the second rainfall prediction result output by inputting the real-time weather data sample ranked as t into the second initial rainfall recognition model.

[0139] In some embodiments, the initial rainfall recognition model is obtained through the following training process:

[0140] Step B1, obtain multiple weather video segments and build an initial model, wherein the initial model includes a lightweight convolutional neural network, a position encoding layer, a deep learning model encoder and a fully connected neural network.

[0141] Step B2, each training process performs the following operations:

[0142] Step B21: input the weather video segment into the initial model, and use the lightweight convolutional neural network to extract features from the weather video data to obtain multiple first feature vectors.

[0143] Step B22: Position-encode the plurality of first feature vectors through the position encoding layer to obtain a plurality of second feature vectors, each of which includes position information.

[0144] Step B23, using the deep learning model encoder to perform attention mechanism calculation based on the multiple second eigenvectors and the position information corresponding to the multiple second eigenvectors to obtain a third eigenvector.

[0145] Step B24, classifying through the fully connected neural network based on the third eigenvector, and outputting the training rainfall prediction result through the initial model.

[0146] Step B3, until a preset number of training times is reached, the trained initial model is used as the initial rainfall recognition model.

[0147] In the above scheme, the collection and annotation process of multiple weather video segments is as follows:

[0148] The DVR camera video recording on the vehicle is used to collect weather video data for a period of time in various scenarios such as rainy and sunny days, daytime and nighttime.

[0149] The annotation is not for a single frame, but for a video segment. The weather video data is segmented according to the rainfall intensity level. For example, the rainfall intensity can be divided into several rainfall intensity levels: no rain, weak, moderate, strong, and extremely strong. The weather video data is segmented, and videos with the same intensity are segmented into a video segment. The label of this video segment is the corresponding rainfall intensity level.

[0150] like Figure 2A , the video segment is intercepted at 2s, 10fps (the specific adjustment can be made according to the algorithm and the amount of calculation, and it is not specifically limited here), and the label of each intercepted weather video data is the label of the video segment. In order to increase the amount of data, a 2s (can be set according to the specific situation, and it is not specifically limited here) window can be slid on the video to intercept the weather video data, that is, each weather video data (i.e. clip) may have an area with overlapping frames, so as to increase the amount of weather video data and further improve the training effect of the initial rainfall recognition model.

[0151] For example, Figure 2B Each weather video segment (i.e. clip) has 20 frames (varies depending on the parameters), and each frame is extracted with mobilenetv2 (a lightweight convolutional neural network). The output of the fully connected layer before the last classification layer of the model is used as the first feature vector. A first feature vector is extracted from each frame, and 20 first feature vectors are obtained. Each first feature vector is called a token embedding (a word segmentation embedding feature, derived from natural language processing), and a learnable class token embedding is added before the first feature vector of the first frame.

[0152] Each token embedding uses positional encoding (position encoding layer) to add position information and obtain 20 second feature vectors.

[0153] Then, after the 20 feature vectors pass through the Transformer Encoder (deep learning model encoder), the model performs attention mechanism calculation, analyzes the connection between multiple second feature vectors with added position information, and obtains 20 third feature vectors.

[0154] Then, the third eigenvector of the first frame is found through the added class token embedding (category segmentation embedding feature), and the third eigenvector is classified into rainfall size through MLP Head (fully connected neural network) to obtain the training rainfall prediction result (ie class), until the preset number of training times is reached, and the initial rainfall recognition model is obtained.

[0155] The initial rainfall recognition model can use the time series information obtained by adding position information and multi-frame features to judge the rainfall amount, which has a higher accuracy rate than single-image recognition.

[0156] Based on the same inventive concept, the application scenario corresponding to the rainfall identification method of the above embodiment is described in detail. Figure 2C As shown, the details are as follows:

[0157] The initial rainfall recognition model is not good enough in terms of generalization ability and accuracy, and needs to be continuously optimized on this basis. The initial rainfall recognition model is deployed on a real vehicle. This vehicle may have a traditional rainfall sensor. The initial rainfall recognition model is not put online, and the prediction results are only used to continuously optimize the initial rainfall recognition model. After optimization, it can be put online to replace the sensor, and can be used for subsequent models without sensors. The overall process is as follows, which is the overall algorithm process of reinforcement learning PPO (Proximal Policy Optimization):

[0158] There are two models on the vehicle, called the P model (with the same structure as the initial rainfall recognition model) and the Q model, where Q is the value function. Q can be a separate model with a similar structure to P. The Q model can also be connected to an fc (fully connected layer) in the P model to predict the value. The second method is preferred here to save computational effort.

[0159] The models saved in the server are P' (the structure is the same as the initial rainfall recognition model) and Q'. Initially, they are the same as the models on the vehicle and on the server (that is, initially P is the same as P', and Q is the same as Q). After subsequent updates, the models on the vehicle and on the server will be different.

[0160] The P model and Q model on the vehicle collect data. The P model on the vehicle outputs the prediction result action (i.e., the rainfall prediction result corresponding to the real-time weather data sample on the vehicle), and the Q model on the vehicle outputs the prediction result value. The actual rainfall level of the sensor or user feedback as the real feedback is used to calculate the reward. The smaller the difference between the model prediction result (i.e., the rainfall prediction result corresponding to the real-time weather data sample on the vehicle) and the real feedback, the greater the reward. The formula is as follows:

[0161] reward=max_class-abs(gt_class-pred_class)

[0162] Among them, max_class is the maximum value of the rainfall intensity level, gt_class is the actual rainfall level, pred_class is the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and reward is the reward value.

[0163] The collected data is transmitted back to the server (P' model and Q' model data storage), and the data content is: video clip (real-time weather data samples sorted as t), model prediction results (i.e., the rainfall prediction results corresponding to the real-time weather data samples on the vehicle), value (i.e., the second rainfall prediction result), and reward (i.e., reward value).

[0164] After collecting data for a period of time (preferably before and after rainy days), these data are used to update P' (i.e., the first initial rainfall recognition model) and Q' (i.e., the second initial rainfall recognition model). The update formula is the update method of PPO (Proximal Policy Optimization, reinforcement learning algorithm), and the Loss formula (i.e., the first loss function):

[0165]

[0166] in:

[0167]

[0168]

[0169] δ t =r t +γV(s t+1 )-V(s t )

[0170] Minimize the above Loss (i.e., the first loss function) to update the P' model (i.e., the first initial rainfall recognition model), and minimize the following formula to update the Q' model (i.e., the second initial rainfall recognition model):

[0171]

[0172] After the P' model and Q' model are updated for a period of time, the online models P and Q are replaced with P'Q'. When the accuracy of the P' model and Q' model after replacing P and Q is improved, the update frequency can be reduced or stopped.

[0173] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0174] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a rainfall identification device.

[0176] refer to Figure 3 , the rainfall identification device comprises:

[0177] An acquisition module 301 is used to acquire weather data to be predicted, where the weather data to be predicted includes real-time video within a period of time;

[0178] The recognition module 302 is used to input the real-time video into a pre-trained rainfall recognition model to obtain the rainfall prediction result of the weather data to be predicted.

[0179] In some embodiments, the rainfall recognition device further includes a rainfall recognition model training module, and the rainfall recognition model training module includes:

[0180] a sample acquisition unit, for acquiring a real-time weather data sample and a rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and determining a reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, wherein the relationship between the reward value and the first difference is negatively correlated, and the first difference is the difference between the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the actual rainfall level;

[0181] A sample input unit, used for inputting the real-time weather data sample into a pre-trained initial rainfall recognition model to obtain a rainfall prediction result corresponding to the real-time weather data sample;

[0182] The model adjustment unit is used to construct a loss function using a reinforcement learning algorithm based on the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and the reward value corresponding to the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and adjust the initial rainfall recognition model using the loss function to obtain a rainfall recognition model.

[0183] In some embodiments, the initial rainfall recognition model includes a first initial rainfall recognition model and a second initial rainfall recognition model, the real-time weather data sample is subjected to the first initial rainfall recognition model to output a first rainfall prediction result, and the real-time weather data sample is subjected to the second initial rainfall recognition model to output a second rainfall prediction result;

[0184] Model adjustment unit, including:

[0185] A first loss function construction unit is used to construct a first loss function using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle;

[0186] A second loss function construction unit is used to construct a second loss function using a reinforcement learning algorithm based on the second rainfall prediction result and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle;

[0187] A first model adjustment unit, configured to adjust the first initial rainfall recognition model by using the first loss function to obtain an adjusted first initial rainfall recognition model;

[0188] The second model adjustment unit is used to adjust the second initial rainfall recognition model by using the second loss function to obtain an adjusted second initial rainfall recognition model, and use the adjusted first initial rainfall recognition model and the adjusted second initial rainfall recognition model as the rainfall recognition model.

[0189] In some embodiments, the first loss function construction unit includes:

[0190] a ratio calculation subunit, configured to perform ratio calculation based on the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value;

[0191] a calculation processing subunit, configured to perform calculation processing based on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result to obtain a second intermediate value;

[0192] A weighted sum calculation subunit, configured to perform a weighted sum calculation on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value;

[0193] The first loss function construction subunit is used to construct the first loss function based on the first intermediate value and the third intermediate value.

[0194] In some embodiments, the ratio calculation subunit is specifically used to:

[0195] Determine, as a first probability, a probability that a first rainfall prediction result output by the first initial rainfall recognition model of the real-time weather data sample belongs to a target rainfall level;

[0196] Determining, as a second probability, a probability that a rainfall prediction result corresponding to the real-time weather data sample on the vehicle belongs to the target rainfall level;

[0197] A ratio calculation is performed using the first probability and the second probability to obtain a ratio result, and the ratio result is used as the first intermediate value.

[0198] In some embodiments, the computing processing subunit is specifically used to:

[0199] Acquire the time series information corresponding to the real-time weather data sample, and determine, based on the time series information, a next real-time weather data sample corresponding to the next time series information relative to the time series information;

[0200] A summation is performed using the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result corresponding to the next real-time weather data sample to obtain a summation result;

[0201] A difference calculation is performed on the summed result and the second rainfall prediction result corresponding to the real-time weather data sample to obtain a difference result, and the difference result is used as the second intermediate value.

[0202] In some embodiments, the weighted sum calculation subunit is specifically configured to:

[0203] Performing weight matching on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to determine a plurality of weights corresponding to the plurality of second intermediate values;

[0204] According to the multiple weights, weighted summation is performed on the multiple second intermediate values ​​to obtain a weighted summation result, and the weighted summation result is used as the third intermediate value.

[0205] In some embodiments, the second loss function construction unit is specifically used to:

[0206] Performing a decaying cumulative calculation on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle to obtain a decaying cumulative result;

[0207] performing a difference calculation on the second rainfall prediction result and the attenuation accumulation result to obtain a second difference result;

[0208] The second loss function is constructed according to the second difference result.

[0209] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0210] The device of the above embodiment is used to implement the corresponding rainfall identification method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0211] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the rainfall identification method described in any of the above embodiments is implemented.

[0212] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 405. The processor 401, the memory 402, the input / output interface 403, and the communication interface 404 are connected to each other in communication within the device through the bus 405.

[0213] The processor 401 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0214] The memory 402 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 402 and called and executed by the processor 401.

[0215] The input / output interface 403 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0216] The communication interface 404 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0217] The bus 405 comprises a pathway for transmitting information between the various components of the device (eg, the processor 401 , the memory 402 , the input / output interface 403 , and the communication interface 404 ).

[0218] It should be noted that, although the above device only shows the processor 401, the memory 402, the input / output interface 403, the communication interface 404 and the bus 405, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0219] The electronic device of the above embodiment is used to implement the corresponding rainfall identification method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0220] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the rainfall identification method described in any of the above embodiments.

[0221] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0222] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the rainfall identification method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0223] Based on the same inventive concept, this embodiment provides a vehicle corresponding to the rainfall identification device or electronic device or storage medium of any of the above embodiments, and the vehicle is equipped with a rainfall identification device or electronic device or storage medium capable of implementing the rainfall identification method of any of the above embodiments.

[0224] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0225] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0226] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0227] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A rainfall identification method, characterized in that: include: Acquire weather data to be predicted, where the weather data to be predicted includes real-time video within a period of time; The real-time video is input into a pre-trained rainfall recognition model to obtain a rainfall prediction result of the weather data to be predicted.

2. The method according to claim 1, characterized in that The rainfall recognition model is obtained through the following training process, including: Acquire a real-time weather data sample and a rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and determine a reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, wherein the reward value is negatively correlated with a first difference, and the first difference is a difference between the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and an actual rainfall level; Inputting the real-time weather data sample into a pre-trained initial rainfall recognition model to obtain a rainfall prediction result corresponding to the real-time weather data sample; Based on the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples, the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, and the reward values ​​corresponding to the rainfall prediction results corresponding to the real-time weather data samples on the vehicle, a loss function is constructed using a reinforcement learning algorithm, and the initial rainfall recognition model is adjusted using the loss function to obtain a rainfall recognition model.

3. The method according to claim 2, characterized in that The initial rainfall recognition model includes a first initial rainfall recognition model and a second initial rainfall recognition model, the real-time weather data sample outputs a first rainfall prediction result after the first initial rainfall recognition model, and the real-time weather data sample outputs a second rainfall prediction result after the second initial rainfall recognition model; The method comprises: constructing a loss function based on the real-time weather data sample, the rainfall prediction result corresponding to the real-time weather data sample, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle by using a reinforcement learning algorithm, and adjusting the initial rainfall recognition model by using the loss function to obtain a rainfall recognition model, including: Constructing a first loss function using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle; Based on the second rainfall prediction result and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, constructing a second loss function using a reinforcement learning algorithm; Adjusting the first initial rainfall recognition model by using the first loss function to obtain an adjusted first initial rainfall recognition model; The second initial rainfall recognition model is adjusted by using the second loss function to obtain an adjusted second initial rainfall recognition model, and the adjusted first initial rainfall recognition model and the adjusted second initial rainfall recognition model are used as the rainfall recognition model.

4. The method according to claim 3, characterized in that The first loss function is constructed using a reinforcement learning algorithm based on the real-time weather data sample, the first rainfall prediction result, the second rainfall prediction result, the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, and the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, including: Performing a ratio calculation based on the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value; Performing calculation processing based on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result to obtain a second intermediate value; Performing a weighted sum calculation based on multiple second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value; The first loss function is constructed based on the first intermediate value and the third intermediate value.

5. The method according to claim 4, characterized in that The step of performing ratio calculation based on the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the first rainfall prediction result to obtain a first intermediate value includes: Determine, as a first probability, a probability that a first rainfall prediction result output by the first initial rainfall recognition model of the real-time weather data sample belongs to a target rainfall level; Determining, as a second probability, a probability that a rainfall prediction result corresponding to the real-time weather data sample on the vehicle belongs to the target rainfall level; A ratio calculation is performed using the first probability and the second probability to obtain a ratio result, and the ratio result is used as the first intermediate value.

6. The method according to claim 4, characterized in that The step of performing calculations on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result to obtain a second intermediate value includes: Acquire the time series information corresponding to the real-time weather data sample, and determine, based on the time series information, a next real-time weather data sample corresponding to the next time series information relative to the time series information; A summation is performed using the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle and the second rainfall prediction result corresponding to the next real-time weather data sample to obtain a summation result; A difference calculation is performed on the sum result and the second rainfall prediction result corresponding to the real-time weather data sample to obtain a first difference result, and the first difference result is used as the second intermediate value.

7. The method according to claim 4, characterized in that: The step of performing weighted sum calculation on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to obtain a third intermediate value includes: Performing weight matching on a plurality of second intermediate values ​​corresponding to the real-time weather data samples to determine a plurality of weights corresponding to the plurality of second intermediate values; According to the multiple weights, weighted summation is performed on the multiple second intermediate values ​​to obtain a weighted summation result, and the weighted summation result is used as the third intermediate value.

8. The method according to claim 3, characterized in that The second loss function is constructed by using a reinforcement learning algorithm based on the reward value corresponding to the second rainfall prediction result and the rainfall prediction result corresponding to the real-time weather data sample on the vehicle, including: Performing a decaying cumulative calculation on the reward value corresponding to the rainfall prediction result corresponding to the real-time weather data sample on the vehicle to obtain a decaying cumulative result; performing a difference calculation on the second rainfall prediction result and the attenuation accumulation result to obtain a second difference result; The second loss function is constructed according to the second difference result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A vehicle comprising the electronic device according to claim 9.