A training method, system and device for a weather landscape recognition model

Abstract

The present application relates to the field of image processing technology, and discloses a training method, system and device for a weather landscape recognition model. The method includes: preliminarily training an initial recognition model using a sample image data set to obtain an intermediate recognition model; acquiring coordinate information of a reference interference image and reference feature points; splicing the reference interference image and the sample image into a to-be-matched image, and performing feature matching using the coordinate information and feature maps of the to-be-matched image at multiple granularities to obtain a second granularity matching result; determining a similar landscape image according to the second granularity matching result; and fine-tuning the intermediate recognition model using the reference interference image and the similar landscape image to obtain a weather landscape recognition model. The beneficial effect is that the reference interference image is used to automatically search for similar landscape images to construct a fine-tuning training data set, and the model is fine-tuned on the basis of preliminary training, which improves the training efficiency of the model while ensuring the accuracy of the model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method, system, and device for training a weather landscape recognition model. Background Art

[0002] The appearance of weather landscapes is affected by various environmental conditions, and requires the mutual cooperation of multiple environmental conditions including atmospheric conditions, sunlight conditions, weather conditions, terrain conditions, and air pollution conditions. Therefore, it is difficult to predict the appearance and disappearance of weather landscapes. In related technologies, weather landscape pictures used as sample materials usually come from different regions and are taken by different devices at different angles. There is a lack of consistency between sample materials, and the accuracy of classifying and identifying sample materials still needs to be improved. Summary of the Invention

[0003] This application provides a method, system, and device for training a weather landscape recognition model, which solves the technical problem that the weather landscape recognition model has misjudgment results in recognition due to classification errors in the sample image data set, and achieves the technical effects of improving the model training speed and the accuracy of weather landscape recognition.

[0004] To achieve the above object, the main technical solutions adopted in this application include:

[0005] In a first aspect, an embodiment of this application provides a method for training a weather landscape recognition model, and the method includes:

[0006] Preliminarily train an initial recognition model using a sample image data set to obtain an intermediate recognition model; wherein, the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt the weather landscape labels;

[0007] Obtain a reference interference image, and determine the coordinate information of a pre-specified reference feature point in the reference interference image; wherein, the reference interference image has an interference label;

[0008] Stitch the reference interference image and the sample image to obtain a to-be-matched image, perform feature matching using the coordinate information and a first feature map of the to-be-matched image at a first granularity to obtain a first granularity matching result, and perform feature matching using the first granularity matching result, the coordinate information, and a second feature map of the to-be-matched image at a second granularity to obtain a second granularity matching result; wherein, the first granularity is greater than the second granularity;

[0009] If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, setting the interference label for the similar landscape image;

[0010] The intermediate recognition model is fine-tuned using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

[0011] The training method of the weather landscape recognition model proposed in the embodiment of the present application first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0012] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0013] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0014] Optionally, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the step of fine-tuning the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model includes:

[0015] The parameters of the feature extraction part are frozen, and the intermediate recognition model is trained using the reference interference image and the similar landscape image to adjust the parameters of the feature sorting and classification part to obtain the weather landscape recognition model.

[0016] Optionally, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature arrangement and classification part connected in sequence; the initial recognition model is preliminarily trained using the sample image dataset to obtain an intermediate recognition model, including:

[0017] Use the initial feature extraction part to extract features from the sample images in the sample image dataset to obtain sample initial features;

[0018] Use the feature attention part to enhance the main features of the sample initial features to obtain sample enhanced features;

[0019] Use the initial feature arrangement and classification part to perform feature fusion on the sample enhanced features to obtain sample fusion features;

[0020] According to the loss value of the sample fusion features, adjust the parameters of the initial feature extraction part and the initial feature arrangement and classification part until the stop condition is met to obtain the intermediate recognition model.

[0021] Optionally, the stop conditions for the preliminary training of the initial recognition model include a first training stop condition and a second training stop condition, where:

[0022] The first training stop condition is that in a continuous set number of iteration rounds, the accuracy of the initial recognition model in recognizing the test dataset is the same;

[0023] The second training stop condition is that the number of times of traversing the sample image dataset reaches the stop threshold;

[0024] The stop conditions for fine-tuning the intermediate recognition model include a first fine-tuning stop condition and a second fine-tuning stop condition, where:

[0025] The first fine-tuning stop condition is that in a continuous set number of iteration rounds, the accuracy of the intermediate recognition model in recognizing the test dataset is the same;

[0026] The second fine-tuning stop condition is that the number of times of traversing the reference interference images and the similar landscape images reaches the stop threshold.

[0027] Optionally, the first granularity matching result includes a first matching point, and the first matching point is a pixel point in the image to be matched that has a matching relationship with the features of the reference feature point; the second granularity matching result is obtained through the following method:

[0028] Extract image features from the image to be matched at the second granularity to obtain the second feature map;

[0029] Extract the pixel points corresponding to the reference feature points from the second feature map to obtain a first reference feature map; extract the pixel points corresponding to the first matching points from the second feature map to obtain a first sample feature map; splice the first reference feature map and the first sample feature map to obtain a feature point matching image;

[0030] Perform feature matching based on the feature point matching image and the coordinate information of the reference feature points to obtain the second granularity matching result; wherein, the second granularity matching result includes second matching points, and the second matching points are pixel points in the feature point matching image that have a matching relationship with the features of the reference feature points.

[0031] Optionally, obtain the first granularity matching result through the following method:

[0032] Extract image features from the image to be matched at the first granularity to obtain the first feature map;

[0033] Perform feature matching on the image features of the sample image in the first feature map according to the coordinate information of the reference feature points to obtain the first matching points.

[0034] Optionally, determine whether the sample image in the image to be matched is the similar landscape image through the following method:

[0035] If the number of the second matching points exceeds a preset threshold, it is considered that there are the same reference interference items in the sample image and the reference interference image, and it is determined that the sample image is the similar landscape image.

[0036] Optionally, obtain the sample image dataset through the following method:

[0037] Preprocess the collected original sample pictures, and obtain the sample image dataset according to the preprocessed original sample pictures; wherein, the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement;

[0038] The color space enhancement includes:

[0039] Obtain a weather landscape color mask according to the color data of the original sample picture and the weather landscape color range; wherein, the weather landscape color range is the color range corresponding to the weather landscape in the original sample picture;

[0040] Perform color enhancement processing on the original sample picture by using the weather landscape color mask to obtain a color enhanced image;

[0041] Smooth the color-enhanced image to obtain the original sample picture after color space enhancement preprocessing.

[0042] In a second aspect, an embodiment of the present application provides a training system for a weather landscape recognition model. The system includes:

[0043] A preliminary training module for preliminarily training an initial recognition model using a sample image dataset to obtain an intermediate recognition model; wherein, the sample images in the sample image dataset are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt the weather landscape labels.

[0044] An interference reference module for obtaining a reference interference image and determining the coordinate information of a pre-specified reference feature point in the reference interference image; wherein, the reference interference image has an interference label.

[0045] An image matching module for splicing the reference interference image and the sample image to obtain a to-be-matched image, performing feature matching using the coordinate information and a first feature map of the to-be-matched image at a first granularity to obtain a first granularity matching result, and performing feature matching using the first granularity matching result, the coordinate information and a second feature map of the to-be-matched image at a second granularity to obtain a second granularity matching result; wherein, the first granularity is greater than the second granularity.

[0046] An interference determination module for setting the interference label. If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, set the interference label for the similar landscape image.

[0047] A fine-tuning training module for fine-tuning the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

[0048] In a third aspect, an embodiment of the present application provides a computer device, including: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the method according to any one of the above embodiments.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the method according to any one of the above embodiments.

[0050] Fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions, and the computer instructions are used to cause a computer to execute the method described in any one of the above embodiments. Description of the Drawings

[0051] In order to more clearly illustrate the specific implementation manners of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required to be used in the specific implementation manners or the description of the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0052] Figure 1 It is a step diagram of the training method of the weather landscape recognition model provided by the embodiment of the present application;

[0053] Figure 2 It is a schematic diagram of the weather landscape image and the sample interference image in the embodiment of the present application;

[0054] Figure 3 It is a step diagram of fine-tuning the intermediate recognition model in the embodiment of the present application;

[0055] Figure 4 It is a step diagram of training the initial recognition model in the embodiment of the present application;

[0056] Figure 5 It is a step diagram of feature matching on the second feature map in the embodiment of the present application;

[0057] Figure 6 It is a step diagram of feature matching on the first feature map in the embodiment of the present application;

[0058] Figure 7 It is a step diagram of performing color space enhancement in the embodiment of the present application;

[0059] Figure 8 It is a schematic diagram of the change of the loss value and accuracy of the recognition model during the training process in the embodiment of the present application;

[0060] Figure 9 It is a module diagram of the training system of the weather landscape recognition model provided by the embodiment of the present application;

[0061] Figure 10 It is a schematic diagram of the structure of a computer device provided by the embodiment of the present application. Detailed Embodiments

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0063] Different regions have different geographical and climatic conditions, resulting in different weather landscape resources in different regions. The emergence conditions of weather landscapes are very harsh. Usually, multiple environmental conditions, including atmospheric conditions, sunlight conditions, weather conditions, terrain conditions, and air pollution conditions, need to cooperate with each other, making it difficult to accurately predict the emergence and disappearance of weather landscapes. In addition, the duration of weather landscapes is also closely related to environmental conditions. Some weather landscapes often last for a short time, thus posing certain requirements for the speed of weather landscape prediction.

[0064] In related technologies, a convolutional neural network based on deep learning is usually used to perform feature learning on sample images containing weather landscapes to obtain an identification model for identifying weather landscapes. In actual scenarios, sample images are usually collected by different people using different devices, resulting in a lack of consistency in the sources, shooting angles, and clarity of sample pictures. Moreover, the identification and classification of sample pictures are usually completed manually. For weather landscapes with short durations, it is easy to make classification errors when classifying sample pictures in a short time, thereby affecting the identification accuracy of the identification model.

[0065] Based on the above problems, this application proposes a training method, system, and device for a weather landscape identification model. The method includes: preliminarily training an initial identification model using a sample image dataset to obtain an intermediate identification model; performing feature matching on sample images based on reference interference images to determine similar landscape images in the sample image dataset; and fine-tuning the intermediate identification model using the reference interference images and the similar landscape images to obtain a weather landscape identification model.

[0066] Among them, performing feature matching on the sample image based on the reference interference image includes: splicing the reference interference image with the sample image to obtain the image to be matched; performing feature matching using the coordinate information of the reference feature point and the first feature map of the image to be matched at the first granularity to obtain the first granularity matching result; performing feature matching using the first granularity matching result, the coordinate information and the second feature map of the image to be matched at the second granularity to obtain the second granularity matching result. Determining similar landscape images in the sample image data set includes: if the sample image in the image to be matched is determined to be a similar landscape image according to the second granularity matching result, setting the same interference label as the reference interference image for the similar landscape image.

[0067] The training method of the weather landscape recognition model proposed in the embodiment of the present application first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0068] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0069] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0070] The training method of the weather landscape recognition model provided in this specification can be applied to train neural network models, target detection models, transfer learning models, multi-modal learning models, etc. to recognize different weather landscapes. It should be noted that during one training process, the weather landscapes targeted are from the same region; for weather landscapes in different regions, the recognition model can be trained using the corresponding sample image datasets. It can be understood that the object of training can also be machine learning algorithms, including supervised learning algorithms and deep learning algorithms, etc. After making appropriate modifications to this method, this method can also be used to recognize other target objects besides weather landscapes.

[0071] According to an embodiment of the present application, an embodiment of a training method for a weather landscape recognition model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0072] In this embodiment, a training method for a weather landscape recognition model is provided, which can be used to train the above-mentioned image recognition model to recognize weather landscapes. Refer to Figure 1 As shown, the method includes:

[0073] S100. Use the sample image dataset to preliminarily train the initial recognition model to obtain an intermediate recognition model; wherein, the sample images in the sample image dataset are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt weather landscape labels.

[0074] S200. Obtain a reference interference image and determine the coordinate information of the pre-specified reference feature points in the reference interference image; wherein, the reference interference image has an interference label.

[0075] S300. Stitch the reference interference image with the sample image to obtain a to-be-matched image, perform feature matching using the coordinate information and the first feature map of the to-be-matched image at the first granularity to obtain a first-granularity matching result, and perform feature matching using the first-granularity matching result, the coordinate information and the second feature map of the to-be-matched image at the second granularity to obtain a second-granularity matching result; wherein, the first granularity is greater than the second granularity.

[0076] S400. If it is determined according to the second-granularity matching result that the sample image in the to-be-matched image is a similar landscape image, set an interference label for the similar landscape image.

[0077] S500. Fine-tune the intermediate recognition model using the reference interference image and the similar landscape image to obtain a weather landscape recognition model.

[0078] Specifically, the sample image dataset can be an image set of the sky in any region collected in advance, including weather landscape images or similar landscape images. Among them, the weather landscape image can be an image of the weather landscape in any region, and at least one weather landscape is included in the weather landscape image. The weather landscape image has a weather landscape label, and according to the type of weather landscape included in the weather landscape image, the weather landscape label can also be divided into multiple types, such as a blue sky landscape label and a white cloud landscape label, etc. The similar landscape image can be an image collected simultaneously with the weather landscape image, which contains sample interference items, and the sample interference item is a part of the non-weather landscape in the sample image that has similar characteristics to the weather landscape. When the classification time of the sample image dataset is short or the classification accuracy is not high, the sample image cannot be accurately classified. At this time, the similar landscape image and the weather landscape image will have the same weather landscape label in the sample image dataset. During the model training process, the sample interference items in the similar landscape image will cause the recognition model to learn features irrelevant to the weather landscape, resulting in misjudgment results in the recognition process of the weather landscape recognition model.

[0079] Exemplarily illustrate the similar landscape image. When the acquisition time of the sample image is early morning or evening, the sample image may include street lights, and the street lights have the characteristics of high brightness and high contrast, which are similar to the weather landscape to be recognized. At this time, in the case of low classification accuracy, the sample picture containing street lights is used as a similar landscape picture and has the same weather landscape label as the weather landscape image. During the training process, the recognition model learns from the similar landscape picture containing street lights, and then mistakes the characteristics of the street lights for the characteristics of the weather landscape, resulting in the weather landscape recognition model misjudging the street lights as the weather landscape during the recognition process.

[0080] It should be noted that the sample image dataset also includes sample interference images. The sample interference image can be an image determined to contain sample interference items after classification after the sample image is collected, and it has an interference label. Refer to Figure 2 , Figure 2 FIG. is a schematic diagram of the weather landscape image and the sample interference image in the embodiment of the present application. As shown in the figure, in some embodiments, the weather landscape image can be obtained by panoramically photographing the sky at the same angle at multiple positions in any region, including at least one of weather landscapes such as blue sky, white clouds, morning glow, and evening glow. The sample interference image can be an image collected simultaneously with the weather landscape image, which includes at least one of interference items such as night, cloudy day, and rainy day.

[0081] Specifically, the reference interference image can be an image containing any type of reference interference item and having an interference label, where the reference interference item is an interference item with a determined type and is provided with a plurality of reference feature points. The reference feature points can be the pixel points occupied by the reference interference item in the image and have features capable of characterizing the reference interference item, such as color, brightness, or shape, etc. The type and position of the reference interference item can be determined according to the features of the reference feature points. It can be understood that there can be multiple reference interference images for any type of reference interference item, so as to characterize the features of the reference interference item in different aspects.

[0082] Specifically, the image to be matched can be obtained by splicing the reference interference image and the sample image. In order to determine whether the sample image contains the reference interference item in the reference interference image, feature matching is performed according to the features of the image to be matched at the first granularity and the features at the second granularity, where the first granularity is greater than the second granularity. The larger the granularity, the lower the resolution of the image, and the smaller the granularity, the higher the resolution of the image. Therefore, the first granularity matching result can be the feature matching result of the reference interference image and the sample image at a lower resolution, and the second granularity matching result can be the feature matching result of the reference interference image and the sample image at a higher resolution. The accuracy of the second granularity matching result is greater than that of the first granularity matching result. Both the first granularity matching result and the second granularity matching result characterize the similarity between the sample image and the reference interference image, and as the granularity decreases, the accuracy of the feature matching result gradually improves. During the feature matching process, the feature matching result at each granularity is used as the basis for feature matching at the next granularity, thus improving the efficiency of feature matching.

[0083] Correspondingly, when a high precision requirement is imposed on the feature matching result, a third granularity can also be set, and the third granularity matching result is obtained according to the second granularity matching result, where the third granularity is smaller than the second granularity, and the resolution of the third feature map obtained according to the third granularity is higher than that of the second feature map.

[0084] In the case where the feature matching at the second granularity can meet the precision requirement, if it is determined according to the second granularity feature matching result that the sample image contains the reference interference item, the sample image is determined to be a similar landscape image, and an interference label is set for the similar landscape image. By performing feature matching on all sample images in the sample image dataset, the sample image dataset is further classified, the accuracy of the weather landscape features in the sample image dataset is improved, and the accuracy of the trained weather landscape recognition model is increased.

[0085] In some embodiments, after determining similar landscape images in a sample image dataset, reference interference items of the same type are determined based on the types of sample interference items in the similar landscape images, and reference interference images containing the above reference interference items are added to the sample image dataset to increase the number of features of the sample interference items and enhance the learning ability of the intermediate recognition model.

[0086] The training method of the weather landscape recognition model provided in this embodiment first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0087] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0088] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0089] Reference Figure 3 As shown, as an embodiment of the present application, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the intermediate recognition model is fine-tuned using a reference interference image and a similar landscape image to obtain a weather landscape recognition model, including:

[0090] S510. Freeze the parameters of the feature extraction part, use the reference interference image and similar landscape images to train the intermediate recognition model, adjust the parameters of the feature sorting and classification part, and obtain the weather landscape recognition model.

[0091] Specifically, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part. Among them, the feature extraction part is used to extract features from the sample images, and the feature sorting and classification part sorts and classifies the extracted features and learns the features. It should be noted that when fine-tuning the intermediate recognition model, the reference interference items in the reference interference images and the sample interference items in the similar landscape images are interference items of the same type and have the same features.

[0092] By freezing the parameters of the feature extraction part in the intermediate recognition model, the feature extraction ability of the intermediate recognition model is retained, enabling the intermediate recognition model to learn according to the features of the interference items in the reference interference images and the similar landscape images, and specifically adjusting the parameters of the feature sorting and classification part, thereby obtaining the weather landscape recognition model and effectively improving the efficiency of model training.

[0093] Refer to Figure 4 As shown, as an embodiment of the present application, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part connected in sequence; the initial recognition model is preliminarily trained using the sample image data set to obtain an intermediate recognition model, including:

[0094] S110. Use the initial feature extraction part to extract features from the sample images in the sample image data set to obtain sample initial features.

[0095] S120. Use the feature attention part to strengthen the main features of the sample initial features to obtain sample enhanced features.

[0096] S130. Use the initial feature sorting and classification part to perform feature fusion on the sample enhanced features to obtain sample fusion features.

[0097] S140. According to the loss value of the sample fusion features, adjust the parameters of the initial feature extraction part and the initial feature sorting and classification part until the stop condition is met to obtain the intermediate recognition model.

[0098] Specifically, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part, and the above three are connected in sequence. Among them, the initial feature extraction part is used to extract features from the sample images to obtain sample initial features; the feature attention part is used to analyze the sample initial features and strengthen the theme features of the part representing the main body of the image to obtain sample enhanced features; the initial feature sorting and classification part is used to integrate according to the sample enhanced features to obtain sample fusion features.

[0099] Furthermore, the feature attention part adopts an attention mechanism to determine the position of the main subject in the sample image based on the initial features of the sample, and enhances the features corresponding to the main subject, thereby highlighting the main subject in the sample image and improving the learning ability of the initial recognition model.

[0100] In some embodiments, the initial recognition model can be a ResNet-50 model. The ResNet-50 model is a residual convolutional neural network model that adds residual blocks to the traditional convolutional neural network, directly transmitting the data of any layer to the subsequent layer through the residual blocks, thereby overcoming the problems of reduced learning efficiency and inability to improve the model recognition accuracy due to the increase in network depth. The ResNet-50 model includes 49 convolutional layers and 1 fully connected layer. Among them, the convolutional layers are used to extract features from the input sample image, such as edges, colors, and textures, etc.; the fully connected layer is connected to the convolutional layers and is used to integrate the features extracted by the convolutional layers to perform tasks such as classification or regression.

[0101] Furthermore, after the sample image is input into the ResNet-50 model, the sample image first passes through a 7×7 convolutional layer with a stride of 2 and a max pooling layer. The max pooling layer divides and reduces the dimensions of the features extracted by the convolutional layer to obtain a pooled feature map. Secondly, the pooled feature map passes through 16 consecutive residual blocks in sequence for feature extraction to obtain a residual feature map, and the residual feature map is converted into a feature vector of a fixed size through a global average pooling layer. Finally, the feature vector is mapped to the label of the sample image through the fully connected layer to predict and identify the type of the main subject in the sample image.

[0102] Furthermore, according to the difference between the output recognition result and the label of the sample image, the loss value is calculated, and the parameters in the initial feature extraction part and the initial feature sorting and classification part are adjusted according to the loss value until the stop condition is met, thereby obtaining an intermediate recognition model.

[0103] As an embodiment of the present application, the stop conditions for the preliminary training of the initial recognition model include a first training stop condition and a second training stop condition, where:

[0104] The first training stop condition is that in consecutive sets of iteration rounds, the accuracy of the initial recognition model in recognizing the test data set is the same.

[0105] The second training stop condition is that the number of times of traversing the sample image data set reaches the stop threshold.

[0106] The stop conditions for fine-tuning the intermediate recognition model include a first fine-tuning stop condition and a second fine-tuning stop condition, where:

[0107] The first fine-tuning stop condition is that in consecutive sets of iteration rounds, the accuracy of the intermediate recognition model in recognizing the test data set is the same.

[0108] The second fine-tuning stop condition is that the number of times of traversing the reference interference images and similar landscape images reaches the stop threshold.

[0109] Specifically, the test data set can be collected simultaneously with the sample image data set and contains multiple test images. The image main body in the test images is pre-recognized and determined. The accuracy of the initial recognition model or the intermediate recognition model can be tested through the test data set.

[0110] Furthermore, during the training of the initial recognition model, the accuracy of the initial recognition model gradually improves as the number of iteration rounds increases. If the accuracy of the initial recognition model remains the same in consecutive sets of iteration rounds, that is, the iterative training cannot improve the accuracy, then the preliminary training of the initial recognition model is stopped. Similarly, if during the fine-tuning of the intermediate recognition model, the accuracy of the intermediate recognition model remains the same in consecutive sets of iteration rounds, then the fine-tuning of the intermediate recognition model is stopped.

[0111] Specifically, the performance of the intermediate recognition model is related to hyperparameters such as the learning rate strategy or the training cycle during training. Different combinations of hyperparameters correspond to different batch sizes. During training, the initial recognition model randomly selects sample images from the sample image data set according to the batch size for learning until all the sample images in the sample image data set are traversed. If during the training process, the number of times of traversing the sample image data set reaches the stop threshold, it means that the initial recognition model has learned the sample image data set sufficiently. To avoid overfitting, the training of the initial recognition model can be stopped.

[0112] Similarly, during the fine-tuning of the intermediate recognition model, the intermediate recognition model needs to learn the reference interference images and similar landscape images. If the number of times of traversing the reference interference images and similar landscape images reaches the stop threshold, it means that the intermediate recognition model has learned the reference interference images and similar landscape image sets sufficiently. To avoid overfitting, the fine-tuning of the intermediate recognition model can be stopped.

[0113] Refer to Figure 5 As shown, as an embodiment of the present application, the first granularity matching result includes a first matching point, and the first matching point is a pixel point in the image to be matched that has a matching relationship with the feature of the reference feature point; the second granularity matching result is obtained by the following method:

[0114] S310. Extract features from the image to be matched at the second granularity to obtain a second feature map.

[0115] S320. Extract the pixel points corresponding to the reference feature points from the second feature map to obtain the first reference feature map; extract the pixel points corresponding to the first matching points from the second feature map to obtain the first sample feature map; splice the first reference feature map and the first sample feature map to obtain the feature point matching image.

[0116] S330. Perform feature matching based on the feature point matching image and the coordinate information of the reference feature points to obtain the second granularity matching result; wherein, the second granularity matching result includes second matching points, and the second matching points are pixel points in the feature point matching image that have a matching relationship with the features of the reference feature points.

[0117] Specifically, since the first granularity is greater than the second granularity, the resolution of the first feature map is lower, which contains more comprehensive feature information and can describe the features of the reference interference image and the sample image globally; while the resolution of the second feature map is higher, and the features contained therein are more precise, which can describe the features of the reference interference image and the sample image in detail. After pixel point extraction, the first reference feature map only contains the features around the reference feature points, and the first sample feature map only contains the features around the first matching points, thereby effectively removing the features in the second feature map that are irrelevant to the reference feature points and the first matching points, reducing the computational complexity required for feature matching, and then being able to determine the second matching points faster and improving the efficiency of feature matching.

[0118] This method effectively shortens the time required to determine the target matching points by using the features of the image to be matched at multiple granularities for progressive search, improves the speed of searching for similar landscape images through the reference interference picture, thereby shortening the time for constructing the training dataset for fine-tuning, and enhancing the training speed of the weather landscape recognition model.

[0119] It can be understood that the first reference feature map can be extracted from the reference interference image, and the first sample feature map can be extracted from the sample image. Splice the first reference feature map and the first sample feature map to obtain the feature point matching image. The first reference feature map and the first sample feature map can also be directly extracted from the second feature map respectively to obtain the feature point matching image.

[0120] Furthermore, based on the second granularity matching result, similar landscape images in the sample image dataset can be searched, and by setting interference labels for the similar landscape images, the similar landscape images can be distinguished from the weather landscape pictures. Compared with the related technology of manually selecting sample images to construct the training dataset for fine-tuning, this method automatically searches for similar landscape images in the sample image dataset using the reference interference image, improves the accuracy of the sample pictures in the sample image dataset, and thus improves the accuracy of the weather landscape recognition model.

[0121] Refer toFigure 6 As shown in the figure, as an embodiment of the present application, the first granularity matching result is obtained through the following method:

[0122] S340. Extract image features of the image to be matched at the first granularity to obtain a first feature map.

[0123] S350. According to the coordinate information of the reference feature points, perform feature matching on the image features of the sample image in the first feature map to obtain first matching points.

[0124] Specifically, during the feature matching process, determine the position of the reference feature points in the first feature map according to the coordinate information of the reference feature points, and extract the image features of the pixel points at the corresponding positions. According to the above image features, perform matching in the part of the first feature map regarding the sample image, so as to determine the pixel points with image features matching the above, in order to obtain the first matching points.

[0125] As an embodiment of the present application, the following method is used to determine whether the sample image in the image to be matched is a similar landscape image:

[0126] S360. If the number of second matching points exceeds a preset threshold, it is considered that there are the same reference interference items in the sample image and the reference interference image, and the sample image is determined to be a similar landscape image.

[0127] Specifically, if the number of second matching points exceeds the preset threshold, it means that there are multiple pixel points in the sample image whose features are similar to those of the reference interference item. According to the number of second matching points, it can be determined that there are the same reference interference items in the sample image and the reference interference image, and the sample image is determined to be a similar landscape image. Similar landscape images in the sample image dataset will cause the initial recognition model to learn incorrect features, resulting in misjudgment results during the recognition process of the weather landscape recognition model. This method automatically searches for similar landscape images using the reference interference image, improves the accuracy of the sample pictures in the sample image dataset, optimizes the fine-tuning effect on the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0128] In some embodiments, after determining the similar landscape pictures, determine the reference interference pictures containing the reference interference items of the same type according to the type of the sample interference items therein, and use these reference interference pictures as sample pictures to be added to the sample image dataset, so as to increase the number of interference item features, enhance the fine-tuning effect on the intermediate recognition model, and thus improve the accuracy of the weather landscape recognition model.

[0129] Refer to Figure 7 As shown in the figure, as an embodiment of the present application, the sample image dataset is obtained through the following method:

[0130] Preprocess the collected original sample images, and obtain a sample image dataset based on the preprocessed original sample images; wherein, the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement.

[0131] The color space enhancement includes:

[0132] S620. Obtain a weather landscape color mask according to the color data of the original sample image and the weather landscape color range; wherein, the weather landscape color range is the color range corresponding to the weather landscape in the original sample image.

[0133] S630. Perform color enhancement processing on the original sample image using the weather landscape color mask to obtain a color-enhanced image.

[0134] S640. Smooth the color-enhanced image to obtain the original sample image after color space enhancement preprocessing.

[0135] Specifically, the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement. Among them, normalization can adjust the features of the sample images to the same range, improving the stability of model training; size adjustment can adjust the sample images to the same size, reducing the interference of image size on model training; contrast adjustment can enhance the contrast difference between the image main body and the background in the sample image, thereby highlighting the image main body.

[0136] Furthermore, color space enhancement can be used to enhance the color of the image main body in the sample image, thereby strengthening the features of the image main body. In color space enhancement, first determine the weather landscape color mask according to the color data of the original sample image and the weather landscape color range, wherein the weather landscape color range is determined in advance according to the weather landscape to be recognized. According to the weather landscape color range, the pixel points in the original sample image whose colors are within the weather landscape color range are regarded as the weather landscape, thereby obtaining the weather landscape color mask for the weather landscape.

[0137] Secondly, use the weather landscape color mask to perform color enhancement processing on the weather landscape in the original sample image to obtain a color-enhanced image. Specifically, the color enhancement processing includes weighting the color of the weather landscape to increase the weight of the color corresponding to the weather landscape, thereby highlighting the color of the weather landscape in the original sample image.

[0138] Finally, the color-enhanced image is smoothed to reduce the color discontinuity after color enhancement processing, and the brightness and contrast of the color-enhanced image are further adjusted to obtain the original sample picture after preprocessing of color space enhancement. By performing color space enhancement on the original sample picture, the weather landscape that is the main body of the image in the original sample picture is made more prominent, thereby improving the learning ability of the initial recognition model.

[0139] The following introduces an actual scenario of using the training method of the weather landscape recognition model provided by this application. The sample image data used in this actual scenario are historical observation pictures obtained through a weather phenomenon video intelligent observer for multiple national-level station locations in the Beijing area. Among them, the weather phenomenon video intelligent observer uses a fixed fish-eye lens and can capture a circular area within 360 degrees of the sky. The multiple national-level stations include Shunyi, Haidian, Yanqing, Foyeding, Tanghekou, Huairou, Miyun, Shangdianzi, Pinggu, Tongzhou, Chaoyang, Changping, Zhaitang, Mentougou, Observatory, Shijingshan, Fengtai, Daxing, Fangshan, and Xiayunling. The time range is from August 29, 2020, to November 30, 2021. The acquisition interval of the sample images is 30 minutes. The weather landscapes in the sample pictures include blue sky, white clouds, morning glow, and evening glow. Based on the above sample pictures, a sample image data set is obtained through preprocessing, and a weather landscape recognition model is obtained by training and fine-tuning the ResNet-50 model.

[0140] Refer to Figure 8 , Figure 8 is a schematic diagram of the changes in the loss value and accuracy of the recognition model during the training process of the embodiment of this application. As shown in the figure, during the training process, the training loss value (Train Loss) of the weather landscape recognition model gradually decreases as the number of iteration rounds increases, and the training accuracy (Train Accuracy) gradually increases as the number of iteration rounds increases. Finally, the accuracy of the weather landscape recognition model in recognizing the sample picture data set is 91.72%. The weather landscape recognition model obtained for each iteration round is tested. The test loss value (Test Loss) of the weather landscape recognition model is stable at 0.4, and the test accuracy (Test Accuracy) of the weather landscape recognition model in recognizing the test data set is 89.97%.

[0141] The accuracy of the weather landscape recognition model in recognizing different types of weather landscapes is shown in Table 1. Among them, the accuracy of the weather landscape recognition model in recognizing blue sky and white clouds can reach more than 90%, the accuracy in recognizing evening glow is close to 90%, and the accuracy in recognizing morning glow is slightly lower, close to 80%.

[0142] Table 1 Accuracy of the weather landscape recognition model in recognizing different types of weather landscapes

[0143] Blue Sky Interference Sunrise Glow Sunset Glow White Cloud Average Sample Picture Dataset 96.97％ 91.16％ 87.33％ 94.37％ 87.27％ 91.72％ Test Dataset 95.43％ 90.11％ 79.45％ 89.94％ 92.00％ 89.97％

[0144] Correspondingly, please refer to Figure 9 , an embodiment of the present application provides a training system for a weather landscape recognition model, and the system includes:

[0145] A preliminary training module 100, configured to perform preliminary training on an initial recognition model by using a sample image data set to obtain an intermediate recognition model; wherein, the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt weather landscape labels.

[0146] An interference reference module 200, configured to obtain a reference interference image and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein, the reference interference image has an interference label.

[0147] An image matching module 300, configured to splice the reference interference image and the sample image to obtain a to-be-matched image, perform feature matching by using the coordinate information and a first feature map of the to-be-matched image at a first granularity to obtain a first granularity matching result, and perform feature matching by using the first granularity matching result, the coordinate information and a second feature map of the to-be-matched image at a second granularity to obtain a second granularity matching result; wherein, the first granularity is greater than the second granularity.

[0148] An interference determination module 400, configured to set an interference label, and if it is determined according to the second granularity matching result that the sample image in the to-be-matched image is a similar landscape image, set an interference label for the similar landscape image.

[0149] A fine-tuning training module 500, configured to perform fine-tuning on the intermediate recognition model by using the reference interference image and the similar landscape image to obtain a weather landscape recognition model.

[0150] In some optional implementation manners, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the fine-tuning training module 500 includes:

[0151] A freezing and fine-tuning unit, configured to freeze the parameters of the feature extraction part, and perform training on the intermediate recognition model by using the reference interference image and the similar landscape image to adjust the parameters of the feature sorting and classification part to obtain a weather landscape recognition model.

[0152] In some optional implementation manners, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part connected in sequence; the preliminary training module 100 includes:

[0153] An initial feature extraction unit is configured to extract features from the sample images in the sample image dataset by using an initial feature extraction part, so as to obtain sample initial features.

[0154] An initial feature enhancement unit is configured to enhance the main features of the sample initial features by using a feature attention part, so as to obtain sample enhanced features.

[0155] An initial feature fusion unit is configured to fuse the features of the sample enhanced features by using an initial feature sorting and classification part, so as to obtain sample fusion features.

[0156] An initial parameter adjustment unit is configured to adjust the parameters of the initial feature extraction part and the initial feature sorting and classification part according to the loss value of the sample fusion features until a stop condition is met, so as to obtain an intermediate recognition model.

[0157] In some optional embodiments, the stop conditions in the preliminary training module 100 include a first training stop condition and a second training stop condition, where:

[0158] The first training stop condition is that the accuracy of the initial recognition model in recognizing the test dataset is the same in continuously set numbers of iteration rounds.

[0159] The second training stop condition is that the number of times of traversing the sample image dataset reaches a stop threshold.

[0160] The stop conditions in the fine-tuning training module 500 include a first fine-tuning stop condition and a second fine-tuning stop condition, where:

[0161] The first fine-tuning stop condition is that the accuracy of the intermediate recognition model in recognizing the test dataset is the same in continuously set numbers of iteration rounds.

[0162] The second fine-tuning stop condition is that the number of times of traversing the reference interference images and the similar landscape images reaches a stop threshold.

[0163] In some optional embodiments, the first granularity matching result includes a first matching point, and the first matching point is a pixel point in the image to be matched that has a matching relationship with the features of the reference feature points; the image matching module 300 includes:

[0164] A second feature extraction unit is configured to extract image features from the image to be matched at a second granularity, so as to obtain a second feature map.

[0165] A feature extraction unit is configured to extract the pixel points corresponding to the reference feature points in the second feature map to obtain a first reference feature map; extract the pixel points corresponding to the first matching points in the second feature map to obtain a first sample feature map; splice the first reference feature map and the first sample feature map to obtain a feature point matching image.

[0166] A second granularity matching unit, configured to perform feature matching based on the coordinate information of the feature point matching image and the reference feature points to obtain a second granularity matching result; wherein, the second granularity matching result includes second matching points, and the second matching points are pixel points in the feature point matching image that have a matching relationship with the features of the reference feature points.

[0167] In some optional embodiments, the image matching module 300 further includes:

[0168] A first feature extraction unit, configured to perform image feature extraction on the image to be matched at a first granularity to obtain a first feature map.

[0169] A first granularity matching unit, configured to perform feature matching on the image features of the sample image in the first feature map according to the coordinate information of the reference feature points to obtain first matching points.

[0170] In some optional embodiments, the image matching module 300 further includes:

[0171] A similar landscape image determination unit, configured to determine whether the sample image is a similar landscape image. If the number of second matching points exceeds a preset threshold, it is considered that there are the same reference interference items in the sample image and the reference interference image, and the sample image is determined to be a similar landscape image.

[0172] In some optional embodiments, the system further includes a sample acquisition module, including:

[0173] A sample preprocessing unit, configured to preprocess the collected original sample pictures, and obtain a sample image dataset according to the preprocessed original sample pictures; wherein, the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement.

[0174] Wherein, the sample preprocessing unit includes:

[0175] A mask determination subunit, configured to obtain a weather landscape color mask according to the color data of the original sample picture and the weather landscape color range; wherein, the weather landscape color range is the color range corresponding to the weather landscape in the original sample picture.

[0176] A color enhancement subunit, configured to perform color enhancement processing on the original sample picture by using the weather landscape color mask to obtain a color enhanced image.

[0177] A smoothing processing subunit, configured to perform smoothing processing on the color enhanced image to obtain the original sample picture after preprocessing of color space enhancement.

[0178] The further function descriptions of the above various modules and units are the same as those in the corresponding above embodiments, and will not be repeated here.

[0179] The training system of the weather landscape recognition model in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit), a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0180] Please refer to Figure 10 , Figure 10 , which is a schematic structural diagram of a computer device provided by an embodiment of the present application. As shown in the figure, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as a server array, a set of blade servers, or a multi-processor system). Figure 10 In

[0181] Processor 10 can be a central processor, a network processor, or a combination thereof. Among them, processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0182] Among them, the memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0183] The memory 20 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the computer device and the like. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely disposed relative to the processor 10, and these remote memories may be connected to the computer device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0184] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memories.

[0185] The computer device further includes a communication interface 30 for communicating the computer device with other devices or communication networks.

[0186] The embodiments of the present application further provide a computer-readable storage medium. The methods according to the embodiments of the present application may be implemented in hardware, firmware, or may be implemented as computer code that can be recorded on a storage medium, or may be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the methods described herein can be stored on such a software process on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0187] The embodiments of the present application provide a computer program product. The computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods of any embodiment of the present application.

[0188] Although embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations fall within the scope defined by the appended claims.

[0189] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0190] For the convenience of description, the above devices are described by dividing them into various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0191] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0193] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1The functions specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one Figure 1 one process or more processes and / or boxes Figure 1 step of the functions specified in one or more boxes.

[0195] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, commodity or device comprising the element.

[0196] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0197] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

[0198] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A training method for a weather landscape recognition model, characterized in that The method includes: Using a sample image dataset to preliminarily train an initial recognition model to obtain an intermediate recognition model; wherein, the sample images in the sample image dataset are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt the weather landscape labels; Obtaining a reference interference image and determining the coordinate information of a pre-specified reference feature point in the reference interference image; wherein, the reference interference image has an interference label; Stitching the reference interference image and the sample image to obtain a to-be-matched image, performing feature matching on the coordinate information and a first feature map of the to-be-matched image at a first granularity to obtain a first granularity matching result, and performing feature matching on the first granularity matching result, the coordinate information and a second feature map of the to-be-matched image at a second granularity to obtain a second granularity matching result; wherein, the first granularity is greater than the second granularity; If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, setting the interference label for the similar landscape image; Using the reference interference image and the similar landscape image to fine-tune the intermediate recognition model to obtain the weather landscape recognition model.

2. The method according to claim 1, wherein The intermediate recognition model includes a feature extraction part and a feature sorting and classification part; The using the reference interference image and the similar landscape image to fine-tune the intermediate recognition model to obtain the weather landscape recognition model includes: Freezing the parameters of the feature extraction part, and using the reference interference image and the similar landscape image to train the intermediate recognition model to adjust the parameters of the feature sorting and classification part to obtain the weather landscape recognition model.

3. The method according to claim 1, characterized in that The initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part connected in sequence; The using a sample image dataset to preliminarily train an initial recognition model to obtain an intermediate recognition model includes: Using the initial feature extraction part to extract features from the sample images in the sample image dataset to obtain sample initial features; Using the feature attention part to strengthen the main features of the sample initial features to obtain sample enhanced features; Using the initial feature sorting and classification part to perform feature fusion on the sample enhanced features to obtain sample fusion features; According to the loss value of the sample fusion features, adjusting the parameters of the initial feature extraction part and the initial feature sorting and classification part until the stop condition is met to obtain the intermediate recognition model.

4. The method according to claim 3, wherein The stop conditions for the preliminary training of the initial recognition model include a first training stop condition and a second training stop condition, wherein: The first training stop condition is that in a continuous set number of iteration rounds, the recognition accuracy of the initial recognition model for the test dataset is the same; The second training stop condition is that the number of times of traversing the sample image dataset reaches the stop threshold; The stopping conditions for fine-tuning the intermediate recognition model include a first fine-tuning stopping condition and a second fine-tuning stopping condition, where: The first fine-tuning stopping condition is that in consecutive sets of iterative rounds, the accuracy of the intermediate recognition model in recognizing the test data set is the same; The second fine-tuning stopping condition is that the number of times of traversing the reference interference image and the similar landscape image reaches the stopping threshold.

5. The method according to claim 1, characterized in that, The first granularity matching result includes first matching points, which are pixel points in the image to be matched that have a matching relationship with the features of the reference feature points; The second granularity matching result is obtained by the following method: Extract image features of the image to be matched at the second granularity to obtain the second feature map; Extract the pixel points corresponding to the reference feature points in the second feature map to obtain the first reference feature map; Extract the pixel points corresponding to the first matching points in the second feature map to obtain the first sample feature map; splice the first reference feature map and the first sample feature map to obtain a feature point matching image; Perform feature matching according to the coordinate information of the feature point matching image and the reference feature points to obtain the second granularity matching result; where the second granularity matching result includes second matching points, which are pixel points in the feature point matching image that have a matching relationship with the features of the reference feature points.

6. The method according to claim 5, wherein The first granularity matching result is obtained by the following method: Extract image features of the image to be matched at the first granularity to obtain the first feature map; Perform feature matching on the image features of the sample image in the first feature map according to the coordinate information of the reference feature points to obtain the first matching points.

7. The method according to claim 5, wherein Judge whether the sample image in the image to be matched is the similar landscape image in the following manner: If the number of the second matching points exceeds the preset threshold, it is considered that there are the same reference interference items in the sample image and the reference interference image, and it is determined that the sample image is the similar landscape image.

8. The method according to any one of claims 1 to 7, characterized in that The sample image data set is obtained by the following method: Preprocess the collected original sample pictures, and obtain the sample image data set according to the preprocessed original sample pictures; where the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement; The color space enhancement includes: Obtain a weather landscape color mask according to the color data of the original sample picture and the weather landscape color range; where the weather landscape color range is the color range corresponding to the weather landscape in the original sample picture; Perform color enhancement processing on the original sample picture using the weather landscape color mask to obtain a color-enhanced image; Perform smoothing processing on the color-enhanced image to obtain the original sample picture after preprocessing of color space enhancement.

9. A training system for a weather landscape recognition model, characterized in that, The system includes: A preliminary training module for preliminarily training an initial recognition model using a sample image dataset to obtain an intermediate recognition model; wherein, the sample images in the sample image dataset are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images adopt the weather landscape labels; An interference reference module for obtaining a reference interference image and determining coordinate information of pre-specified reference feature points in the reference interference image; wherein, the reference interference image has an interference label; An image matching module for splicing the reference interference image and the sample image to obtain a to-be-matched image, performing feature matching using the coordinate information and a first feature map of the to-be-matched image at a first granularity to obtain a first granularity matching result, and performing feature matching using the first granularity matching result, the coordinate information and a second feature map of the to-be-matched image at a second granularity to obtain a second granularity matching result; wherein, the first granularity is greater than the second granularity; An interference determination module for setting the interference label, and if it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, setting the interference label for the similar landscape image; A fine-tuning training module for fine-tuning the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

10. A computer device, characterized in that, Comprising: A memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the method according to any one of claims 1 to 8.

Images