Neural network updating method and image processing method
By updating the batch normalization layer parameters of existing neural network models, using the target sample set to adapt to new weather scenarios, the problem of poor inference ability of autonomous driving car models in diverse weather scenarios is solved, and the model is quickly improved in new weather scenarios is achieved Accuracy and efficiency in the scenario.
Patent Information
- Application Number
- CN202310591575.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
When facing diverse weather scenarios, insufficient training data leads to poor inference capabilities, and existing methods require a lot of resources and time to retrain models.
By obtaining the target sample sets in different weather scenarios, the existing neural network models are updated, the parameters of the batch normalization layer are adjusted, and the target neural network models adapted to the new weather scenarios are generated to avoid retraining the model.
Without retraining the model, the model's reasoning ability and accuracy of the new weather scenarios are improved, saving manpower and material costs.
Smart Images

Figure CN119027752A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of automobile technology, and in particular to a neural network updating method and an image processing method. Background Art
[0002] The weather environment has a great impact on the autonomous driving of cars. In the model training under the autonomous driving scenario, the training data usually only contains sampled data under one or a few weather scenarios. However, the weather faced by the actual operation of the vehicle is much more complicated, resulting in large deviations in the model's reasoning capabilities such as recognition, detection or classification in actual applications. When faced with new weather scenarios, the reasoning ability is poor and the accuracy is low. If a complete model training process is carried out for each weather scenario, it will inevitably consume a lot of processing resources and take a long time to train. Therefore, how to quickly complete the training of the corresponding model for a new weather scenario has become an urgent problem to be solved. Summary of the invention
[0003] In order to solve the above technical problems, the present disclosure provides a neural network updating method and an image processing method to improve the model's reasoning ability in processing new weather scene data.
[0004] The present disclosure provides a method for updating a neural network, comprising:
[0005] Obtain the trained original neural network model corresponding to the original weather scene;
[0006] Acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes;
[0007] The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the target weather scene.
[0008] In some embodiments, updating the original neural network model according to the target sample set includes:
[0009] Determine the original statistical value of the characteristic value of the sample data in the target sample set;
[0010] Key parameters of a batch normalization layer in the original neural network model are updated based on the original statistical values.
[0011] In some embodiments, the key parameters include batch mean and batch variance;
[0012] The updating of key parameters of a batch normalization layer in the original neural network model based on the original statistical values includes:
[0013] Get the historical batch mean and historical batch variance of the batch normalization layer of the previous batch;
[0014] According to the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch mean;
[0015] According to the historical batch variance and the original variance in the original statistical value, the batch variance of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch variance.
[0016] In some embodiments, updating the batch mean of the batch normalization layer of the current batch in the original neural network model according to the historical batch mean and the original mean in the original statistical value to obtain the current batch mean includes:
[0017] According to a preset mean update formula and the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain a current batch mean;
[0018] Wherein, the mean update formula includes:
[0019]
[0020] in, represents the current batch mean, represents the historical batch mean, μ k represents the original mean, ζ is a constant, ρ k represents the control coefficient of momentum update, ρ k = k-1 ·, ω is a constant, and k represents the current batch.
[0021] In some embodiments, the updating of the original neural network model according to the target sample set to obtain a target neural network model matching the target weather scene includes:
[0022] The original neural network model is updated respectively according to each group of the target sample sets to obtain multiple target neural network models that match each of the target weather scenes one by one.
[0023] In some embodiments, the updating of the original neural network model according to the target sample set to obtain a target neural network model matching the target weather scene includes:
[0024] Each of the target sample sets is used as a current sample set one by one, and the following update operations are performed based on the current sample set:
[0025] The original neural network model is updated according to the current sample set to obtain a new neural network model;
[0026] Determine a new current sample set from the remaining multiple groups of the target sample sets, and repeat the above updating operation based on the new current sample set;
[0027] When all the target sample sets are processed, the updating operation is stopped to obtain a target neural network model matching the multiple target weather scenes.
[0028] In some embodiments, the method further comprises:
[0029] The target neural network model is used to process the data to be tested under the target weather scenario.
[0030] In some embodiments, obtaining at least one target sample set includes:
[0031] For any set of test data sets under the target weather scenario, extract a plurality of original samples from the test data sets according to a preset extraction ratio;
[0032] Performing data enhancement processing on the plurality of original samples to generate a target sample set; wherein the data enhancement processing includes: flipping, rotating and / or cropping.
[0033] The present disclosure provides an image processing method, comprising:
[0034] Determine the current weather scene and collect the target image under the current weather scene;
[0035] Obtain the trained original neural network model corresponding to the original weather scene;
[0036] Acquire a target sample set corresponding to the current weather scene;
[0037] The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the current weather scene;
[0038] The target image is processed by the target neural network model.
[0039] The present disclosure provides a neural network updating device, comprising:
[0040] A first model acquisition module is used to obtain a trained original neural network model corresponding to the original weather scene;
[0041] A first sample acquisition module, configured to acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes;
[0042] The first model updating module is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the target weather scene.
[0043] The present disclosure provides an image processing device, comprising:
[0044] An image acquisition module, used to determine the current weather scene and acquire a target image under the current weather scene;
[0045] The second model acquisition module is used to obtain the trained original neural network model corresponding to the original weather scene;
[0046] A second sample acquisition module, used to acquire a target sample set corresponding to the current weather scene;
[0047] A second model updating module is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the current weather scene;
[0048] An image processing module is used to process the target image through the target neural network model.
[0049] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute the steps of any of the above methods.
[0050] The present disclosure also provides an electronic device, comprising:
[0051] one or more processors;
[0052] A memory for storing one or more programs or instructions;
[0053] The processor is used to execute the steps of any of the above methods by calling the program or instruction stored in the memory.
[0054] The present disclosure also provides a vehicle, comprising the above-mentioned neural network updating device or the above-mentioned image processing device.
[0055] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0056] In the technical solution provided by the embodiment of the present disclosure, the original neural network model is adapted to the original weather scene, and the target sample set corresponds to a target weather scene that is different from the original weather scene; thus, the original neural network model is updated according to the target sample set, and the obtained target neural network model can adapt to both the original weather scene and the new target weather scene. Therefore, when the target weather scene in the actual application is inconsistent with the original weather scene during model training, this solution can use the target sample set under the target weather scene to update the model without retraining the model, quickly improve the model's reasoning ability for data under the new weather scene, and improve the model's reasoning ability and accuracy in the new weather scene. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0059] Figure 1 A flowchart of a neural network updating method provided by an embodiment of the present disclosure;
[0060] Figure 2 A schematic diagram of a model updating method 1 provided in an embodiment of the present disclosure;
[0061] Figure 3 A schematic diagram of a second model updating method provided in an embodiment of the present disclosure;
[0062] Figure 4 A structural block diagram of a neural network updating device provided in an embodiment of the present disclosure;
[0063] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0064] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0066] For autonomous driving, the weather environment that the vehicle faces in actual operation is far more complex than the weather included in the model's training data. When there is a difference between the weather scenes corresponding to the measured data and the training data, the model's reasoning ability will have a large deviation, and the reasoning ability and accuracy will be poor when facing new weather scenes. In response to the above problems, for deep learning neural networks, the model is generally further trained for new weather scenes through manually annotated data, but this requires manpower and time costs. Based on this, the embodiments of the present disclosure provide a neural network update method and an image processing method. The present disclosure updates the original neural network model corresponding to the original weather scene according to the target sample set under the target weather scene. Not only does it not need to retrain the model, but the updated target neural network model can also effectively reason about target weather scenes other than the original weather scene, thereby improving the model's reasoning ability and accuracy in new weather scenes.
[0067] Figure 1 This is a flowchart of a neural network update method provided by an embodiment of the present disclosure. This method is applicable to scenarios where a vehicle model is updated when facing different weather conditions. It can use the target sample set under the new weather scenario to update the trained original neural network model, and can be applied to autonomous vehicles and manned vehicles. This method can be executed by a neural network update device, which can be implemented in software and / or hardware. Figure 1 As shown, the method comprises the following steps:
[0068] Step S102, obtaining the trained original neural network model corresponding to the original weather scene.
[0069] In one example, the original weather scene is a sunny scene; this embodiment collects data in a sunny scene, and the data is generally an image. The neural network model is trained according to the collected data to obtain a trained original neural network model. In a sunny scene, the original neural network model can be used to perform inference such as recognition, classification, detection or segmentation of images collected in actual vehicle operating conditions.
[0070] However, for the original neural network model, the data to be processed in the actual application process often has a large scene difference from the data used for model training; when the original neural network model is used to reason about images from different weather scenes (such as rainy days, foggy days or snowy days, etc.), the performance of the original neural network model will deteriorate due to the differences in weather scenes. In this case, this embodiment can improve the reasoning ability of the model in new weather scenes according to the following steps.
[0071] Step S104, obtaining at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes, such as rain, snow, fog, sand and dust, etc. It can be understood that the sample data in the same set of target sample sets all correspond to the same target weather scene.
[0072] Considering that a large number of test data sets are easily accumulated in practical applications, in order to facilitate the acquisition of the target sample set, this embodiment can acquire the target sample set based on the test data set. In one embodiment, the process of acquiring the target sample set includes:
[0073] For any set of test data sets under the target weather scenario, multiple original samples are extracted from the test data set according to a preset extraction ratio. This scheme randomly extracts multiple original samples from the test data set according to the extraction ratio, which can ensure the data volume and diversity of the original samples. Then, data enhancement processing is performed on the multiple original samples to generate a target sample set; wherein the data enhancement processing includes: flipping, rotating and / or cropping. The data in the test data set is generally an image, so this embodiment can perform data enhancement processing such as flipping, rotating and cropping on each extracted original sample to obtain a richer and more diverse target sample set.
[0074] Step S106, updating the original neural network model according to the target sample set to obtain a target neural network model that matches the target weather scene.
[0075] In the neural network model, the batch normalization (BN) layer is an important component of the model. The batch normalization layer can use the mean and variance of small batch samples to continuously adjust the intermediate output of the neural network model, making the data distribution consistent and making the intermediate output values of each layer of the entire neural network model more stable. The batch normalization layer can increase the classification effect of the model, and the parameter adjustment process of the batch normalization layer is very simple.
[0076] In this case, the method for updating the original neural network model according to the target sample set in this embodiment can be: first determine the original statistical values of the characteristic values of the sample data in the target sample set; and then update the key parameters of the batch normalization layer in the original neural network model based on the original statistical values.
[0077] In a specific implementation, the mean and variance of the characteristic values of the sample data in the target sample set can be counted from at least one dimension such as the spatial dimension or the coordinate dimension to obtain the original statistical value. Taking the coordinate dimension as an example, the height value and width value of each sample data in the target sample set can be counted to obtain the coordinate dimension mean and coordinate dimension variance; the coordinate dimension mean and coordinate dimension variance are the original statistical values.
[0078] Next, the batch normalization layer can be applied between any network layer that can output feature data and the activation layer in the neural network model, so that the batch normalization layer can perform batch normalization operations on the feature data output by the network layer; wherein, the batch normalization operation can be referred to as follows:
[0079]
[0080] In the above formula (1), γ and β represent the scale and offset factors respectively, x is the data sample, E(X) and Var(X) represent the statistics of the expectation and variance of the samples in the entire training set respectively. Represents the output data after batch normalization operation.
[0081] E(X) and Var(X) are determined by the batch mean and batch variance of the target sample set, respectively. Accordingly, the key parameters of the batch normalization layer that need to be updated in this implementation include the batch mean and batch variance.
[0082] According to the above situation, this embodiment can update the batch mean and batch variance of the batch normalization layer in the original neural network model in an offline manner based on the original statistical values, as shown below.
[0083] Get the historical batch mean and historical batch variance of the batch normalization layer for the previous batch.
[0084] According to the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch mean.
[0085] The batch mean of the batch normalization layer is updated in a momentum manner. In a specific example, the batch mean of the batch normalization layer of the current batch in the original neural network model can be updated according to a preset mean update formula and the original mean in the historical batch mean and the original statistical value to obtain the current batch mean;
[0086] Among them, the mean update formula (2) includes:
[0087]
[0088] in, Represents the current batch mean, which can be understood as the updated mean of the current batch. represents the historical batch mean, μ l represents the original mean, ζ is a constant, such as 0.005, ρ k represents the control coefficient of momentum update, whose initial value is, for example, 0.1. During the update process, it can be adjusted according to ρ k = k-1 · Perform dynamic adjustment, ω is a constant, such as 0.94, and k represents the current batch.
[0089] And, based on the historical batch variance and the original variance σ in the original statistics k 2 , update the batch variance of the batch normalization layer of the current batch k in the original neural network model to obtain the current batch variance Among them, the batch variance of the batch normalization layer of the current batch can be updated according to the following variance update formula (3).
[0090]
[0091] It can be understood that the historical batch mean, historical batch variance and control coefficient ρ corresponding to the batch normalization layer of the k-1th batch in the above embodiment are k-1 , which is determined in the same way as the corresponding parameters in the kth batch.
[0092] When the original neural network model needs to be adapted to multiple target weather scenarios, this embodiment may provide the following two methods to update the original neural network model.
[0093] Method 1 is to update a target neural network model for each target weather scene separately. Specifically, the original neural network model is updated separately according to each group of target sample sets to obtain multiple target neural network models that match each target weather scene one by one. Figure 2As shown, assuming that a target neural network model matching the four target weather scenes of rainy day, dusty day, snowy day and foggy day is to be obtained, the original neural network model can be updated according to the target sample set 1 to obtain the target neural network model 1 matching the rainy day weather scene; the original neural network model can be updated according to the target sample set 2 to obtain the target neural network model 2 matching the dusty day weather scene; ... Through this method 1, the target neural network model 1, the target neural network model 2, the target neural network model 3 and the target neural network model 4 matching the four target weather scenes of rainy day, dusty day, snowy day and foggy day can be obtained, and each target weather scene corresponds to its own target neural network model.
[0094] The second method is to update a target neural network model for multiple target weather scenes in a cascade manner. The specific implementation methods include:
[0095] Each target sample set is used as the current sample set one by one, and the following update operation is performed based on the current sample set: the original neural network model is updated according to the current sample set to obtain a new neural network model. Figure 3 As shown, assuming that the first current sample set is the target sample set 1 corresponding to the rainy weather scene, the original neural network model is updated according to the target sample set 1 to obtain a new neural network model 1. Since the new neural network model 1 is based on the original neural network model under the sunny weather scene and is updated using the target sample set 1 under the rainy weather scene, the new neural network model 1 can adapt to data reasoning under both sunny weather scenes and rainy weather scenes. Accordingly, the new neural network model 1 is used as the basis for the next model update.
[0096] A new current sample set is determined from the remaining multiple groups of target sample sets, and the above update operation is repeated based on the new current sample set; for example, the target sample set 2 corresponding to the sandstorm weather scene is used as the new current sample set; at this time, the new neural network model 1 is used as the basis for model update, and the new neural network model 1 is updated according to the target sample set 2 to obtain a new neural network model 2. The new neural network model 2 can not only adapt to the sunny weather scenes and rainy weather scenes that the neural network model 1 has adapted to, but also can adapt to the new sandstorm weather scenes. Accordingly, the new neural network model 2 is used as the basis for the next model update.
[0097] The update operation is continued until all target sample sets are processed and the target neural network model matching multiple target weather scenes is obtained. After the above update operations adapted to each target weather scene, the target neural network model obtained can adapt to the four target weather scenes of rain, dust, snow and fog at the same time.
[0098] In order to adapt to new and different target weather scenarios, multiple target neural network models or one target neural network model can be updated according to the above two methods, providing users with multiple options for actual use.
[0099] After obtaining the target neural network model according to the above embodiment, the method may further include: processing the test data in the target weather scene by the target neural network model. For example, when the vehicle is driving in the rain, in order to ensure driving safety, the target image in the rainy weather scene may be collected, and the pedestrians, bicycles and other targets in the target image may be identified by the target neural network model.
[0100] In summary, the neural network updating method provided by the embodiment of the present disclosure, for the trained original neural network model corresponding to the original weather scene, first obtains the target sample set under the target weather scene, and the target weather scene is different from the original weather scene; then the original neural network model is updated according to the target sample set to obtain the target neural network model that matches the target weather scene. This technical solution uses the original neural network model as the update basis, and can naturally adapt to the original weather scene. At the same time, the model is updated based on the target sample set under the target weather scene, so that the updated target neural network model can adapt to the target weather scene. Therefore, when the target weather scene in the actual application is inconsistent with the original weather scene during model training, this solution can use the target sample set under the target weather scene to update the model without retraining the model, so as to quickly improve the model's reasoning ability for data in the new weather scene, and improve the model's reasoning ability and accuracy in the new weather scene.
[0101] In addition, compared with the existing method of retraining the model by manually annotating data, the present invention can effectively save manpower and material costs by updating the original neural network model according to the target sample set.
[0102] According to the neural network updating method provided in the above embodiment, the present disclosure also provides an image processing method, which may include the following contents:
[0103] First, determine the current weather scene and collect the target image in the current weather scene. In some possible ways, the current scene of the day can be determined based on weather forecast information, or a convolutional neural network can be used to perform image recognition to determine the current weather scene. In actual car driving, it is often necessary to collect images to better assist in intelligent car driving. Thus, in this embodiment, the target image in the current weather scene can be collected by the vehicle's camera; the target image is, for example, an image of objects in front of the vehicle including pedestrians, vehicles, road signs, etc.
[0104] Secondly, obtain the trained original neural network model corresponding to the original weather scene; obtain the target sample set corresponding to the current weather scene; update the original neural network model according to the target sample set to obtain the target neural network model matching the current weather scene.
[0105] The implementation process of this embodiment may refer to the neural network updating method provided in the aforementioned embodiment, and will not be described in detail here.
[0106] Then, the target image is processed by the target neural network model.
[0107] In this embodiment, the processing of the target image by the target neural network model includes: detecting target objects such as pedestrians and vehicles in the target image, identifying the distance between the target object and the vehicle in the target image, detecting the driving status of the driver in the target image, marking dangerous objects in the target image, extracting accident features in the target image, etc., which are not listed one by one here.
[0108] The image processing method provided by the embodiment of the present disclosure obtains a target neural network model that matches the current weather scene according to the above neural network update method, and processes the target image in the current weather scene through the target neural network model. In this solution, the target neural network model is a model that adapts to the current weather scene, and can better process the target image in the current weather scene, thereby improving the accuracy of image processing.
[0109] Corresponding to the method for updating a neural network provided in the embodiment of the present disclosure, the embodiment of the present disclosure also provides a device for updating a neural network. Figure 4 A structural block diagram of a neural network updating device provided in an embodiment of the present disclosure, such as Figure 4 As shown, the updating device of the neural network includes:
[0110] A first model acquisition module 202 is used to acquire a trained original neural network model corresponding to the original weather scene;
[0111] A first sample acquisition module 204 is used to acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes;
[0112] The first model updating module 206 is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the target weather scene.
[0113] In some embodiments, the first model updating module 206 is specifically configured to:
[0114] Determine the original statistical value of the characteristic value of the sample data in the target sample set;
[0115] Key parameters of a batch normalization layer in the original neural network model are updated based on the original statistical values.
[0116] In some embodiments, the key parameters include batch mean and batch variance; the first model updating module 206 is specifically used to:
[0117] The updating of key parameters of a batch normalization layer in the original neural network model based on the original statistical values includes:
[0118] Get the historical batch mean and historical batch variance of the batch normalization layer of the previous batch;
[0119] According to the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch mean;
[0120] According to the historical batch variance and the original variance in the original statistical value, the batch variance of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch variance.
[0121] In some embodiments, the first model updating module 206 is specifically configured to:
[0122] According to a preset mean update formula and the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain a current batch mean;
[0123] Wherein, the mean update formula includes:
[0124]
[0125] in, represents the current batch mean, represents the historical batch mean, μ k represents the original mean, ζ is a constant, ρ k represents the control coefficient of momentum update, ρ k = k-1 ·, ω is a constant, and k represents the current batch.
[0126] In some embodiments, the model updating module 206 is specifically used to:
[0127] The original neural network model is updated respectively according to each group of the target sample sets to obtain multiple target neural network models that match each of the target weather scenes one by one.
[0128] In some embodiments, the first model updating module 206 is specifically configured to:
[0129] Each of the target sample sets is used as a current sample set one by one, and the following update operations are performed based on the current sample set:
[0130] The original neural network model is updated according to the current sample set to obtain a new neural network model;
[0131] Determine a new current sample set from the remaining multiple groups of the target sample sets, and repeat the above updating operation based on the new current sample set;
[0132] When all the target sample sets are processed, the updating operation is stopped to obtain a target neural network model matching the multiple target weather scenes.
[0133] In some embodiments, the updating device of the neural network further includes a data processing module, which is used to:
[0134] The target neural network model is used to process the data to be tested under the target weather scenario.
[0135] In some embodiments, the first sample acquisition module 204 is specifically used to:
[0136] For any set of test data sets under the target weather scenario, extract a plurality of original samples from the test data sets according to a preset extraction ratio;
[0137] Performing data enhancement processing on the plurality of original samples to generate a target sample set; wherein the data enhancement processing includes: flipping, rotating and / or cropping.
[0138] The neural network updating device disclosed in the above embodiments can execute the neural network updating method disclosed in the above embodiments, and has the same or corresponding beneficial effects. To avoid repetition, it will not be described again here.
[0139] Corresponding to the image processing method provided in the embodiment of the present disclosure, the embodiment of the present disclosure also provides an image processing device. The image processing device includes:
[0140] An image acquisition module, used to determine the current weather scene and acquire a target image under the current weather scene;
[0141] The second model acquisition module is used to obtain the trained original neural network model corresponding to the original weather scene;
[0142] A second sample acquisition module, used to acquire a target sample set corresponding to the current weather scene;
[0143] A second model updating module is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the current weather scene;
[0144] An image processing module is used to process the target image through the target neural network model.
[0145] The image processing device disclosed in the above embodiments can execute the image processing method disclosed in the above embodiments, and has the same or corresponding beneficial effects, which will not be described again here to avoid repetition.
[0146] The embodiment of the present disclosure also provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of any of the above-mentioned neural network updating methods:
[0147] Obtain the trained original neural network model corresponding to the original weather scene;
[0148] Acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes;
[0149] The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the target weather scene.
[0150] Or perform the steps of any of the above image processing methods:
[0151] Determine the current weather scene and collect the target image under the current weather scene;
[0152] Obtain the trained original neural network model corresponding to the original weather scene;
[0153] Acquire a target sample set corresponding to the current weather scene;
[0154] The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the current weather scene;
[0155] The target image is processed by the target neural network model.
[0156] Optionally, when executed by a computer processor, the computer executable instructions can also be used to execute the technical solutions of any of the above-mentioned neural network updating methods or image processing methods provided in the embodiments of the present disclosure to achieve corresponding beneficial effects.
[0157] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the embodiments of the present disclosure can be implemented with the help of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present disclosure.
[0158] The embodiment of the present disclosure also provides an electronic device, including: one or more processors; a memory for storing one or more programs or instructions; the processor calls the programs or instructions stored in the memory to execute the steps of any of the above methods to achieve corresponding beneficial effects.
[0159] Figure 5 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure. Figure 5 As shown, the electronic device includes one or more processors 301 and a memory 302 .
[0160] The processor 301 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
[0161] The memory 302 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 301 may run the program instructions to implement the neural network update method, image processing method, and / or other desired functions of the embodiments of the present disclosure described above. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0162] In one example, the electronic device may further include: an input device 303 and an output device 304, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0163] In addition, the input device 303 may also include, for example, a keyboard, a mouse, and the like.
[0164] The output device 304 can output various information to the outside, including the determined distance information, direction information, etc. The output device 304 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0165] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.
[0166] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0167] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for updating a neural network, characterized in that: include: Obtain the trained original neural network model corresponding to the original weather scene; Acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes; The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the target weather scene.
2. The method according to claim 1, characterized in that The updating of the original neural network model according to the target sample set includes: Determine the original statistical value of the characteristic value of the sample data in the target sample set; Key parameters of a batch normalization layer in the original neural network model are updated based on the original statistical values.
3. The method according to claim 2, characterized in that The key parameters include batch mean and batch variance; The updating of key parameters of a batch normalization layer in the original neural network model based on the original statistical values includes: Get the historical batch mean and historical batch variance of the batch normalization layer of the previous batch; According to the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch mean; According to the historical batch variance and the original variance in the original statistical value, the batch variance of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch variance.
4. The method according to claim 3, characterized in that According to the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain the current batch mean, including: According to a preset mean update formula and the historical batch mean and the original mean in the original statistical value, the batch mean of the batch normalization layer of the current batch in the original neural network model is updated to obtain a current batch mean; Wherein, the mean update formula includes: in, represents the current batch mean, represents the historical batch mean, μ k represents the original mean, ζ is a constant, ρ k represents the control coefficient of momentum update, ρ k = k-1 ·, ω is a constant, and k represents the current batch.
5. The method according to claim 1, characterized in that The updating of the original neural network model according to the target sample set to obtain a target neural network model matching the target weather scene includes: The original neural network model is updated respectively according to each group of the target sample sets to obtain multiple target neural network models that match each of the target weather scenes one by one.
6. The method according to claim 1, characterized in that The updating of the original neural network model according to the target sample set to obtain a target neural network model matching the target weather scene includes: Each of the target sample sets is used as a current sample set one by one, and the following update operations are performed based on the current sample set: The original neural network model is updated according to the current sample set to obtain a new neural network model; Determine a new current sample set from the remaining multiple groups of the target sample sets, and repeat the above updating operation based on the new current sample set; When all the target sample sets are processed, the updating operation is stopped to obtain a target neural network model matching the multiple target weather scenes.
7. The method according to claim 1, characterized in that The method further comprises: The target neural network model is used to process the data to be tested under the target weather scenario.
8. The method according to claim 1, characterized in that The obtaining of at least one target sample set comprises: For any set of test data sets under the target weather scenario, extract a plurality of original samples from the test data sets according to a preset extraction ratio; Performing data enhancement processing on the plurality of original samples to generate a target sample set; wherein the data enhancement processing includes: flipping, rotating and / or cropping.
9. An image processing method, characterized in that: include: Determine the current weather scene and collect the target image under the current weather scene; Obtain the trained original neural network model corresponding to the original weather scene; Acquire a target sample set corresponding to the current weather scene; The original neural network model is updated according to the target sample set to obtain a target neural network model that matches the current weather scene; The target image is processed by the target neural network model.
10. A neural network updating device, characterized in that: include: A first model acquisition module is used to obtain a trained original neural network model corresponding to the original weather scene; A first sample acquisition module, configured to acquire at least one set of target sample sets; wherein different target sample sets include sample data under different target weather scenes, and the target weather scenes are different from the original weather scenes; The first model updating module is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the target weather scene.
11. An image processing device, characterized in that: include: An image acquisition module, used to determine the current weather scene and acquire a target image under the current weather scene; The second model acquisition module is used to obtain the trained original neural network model corresponding to the original weather scene; A second sample acquisition module, used to acquire a target sample set corresponding to the current weather scene; A second model updating module is used to update the original neural network model according to the target sample set to obtain a target neural network model that matches the current weather scene; An image processing module is used to process the target image through the target neural network model.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method according to any one of claims 1 to 9.
13. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs or instructions; The processor is used to execute the steps of the method according to any one of claims 1 to 9 by calling the program or instruction stored in the memory.
14. A vehicle, characterized in that: The vehicle includes the updating device of the neural network as claimed in claim 10, or includes the image processing device as claimed in claim 11.
Citation Information
Cited By
Target enhanced fusion perception method, system, device and medium
CN120708012A
