Weather landscape recognition model training method, system and equipment

Abstract

The invention relates to the technical field of image processing, and discloses a training method, system and device for a weather landscape recognition model, and the method comprises the steps: carrying out the preliminary training of an initial recognition model through a sample image data set, and obtaining an intermediate recognition model; obtaining coordinate information of a reference interference image and a reference feature point; splicing the reference interference image and the sample image into a to-be-matched image, and performing feature matching by using the coordinate information and feature maps of the to-be-matched image on multiple granularities to obtain a second granularity matching result; determining a similar landscape image according to the second granularity matching result; and performing fine adjustment on the intermediate identification model by using the reference interference image and the similar landscape image to obtain a weather landscape identification model. The method has the beneficial effects that the similar landscape image is automatically searched by using the reference interference image so as to construct the fine-tuning training data set, and the model is fine-tuned on the basis of preliminary training, so that the model training efficiency is improved on the basis of ensuring the model accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a training method, system and device for a weather landscape recognition model. Background Art

[0002] The appearance of weather landscapes is affected by a variety of environmental conditions, and requires the coordination of multiple environmental conditions, including atmospheric conditions, sunlight conditions, weather conditions, terrain conditions, and air pollution conditions. Therefore, it is difficult to predict the appearance and disappearance of weather landscapes. In related technologies, weather landscape images as sample data usually come from different regions and are taken by different equipment at different angles. There is a lack of consistency between the sample data, and the accuracy of classifying and identifying the sample data still needs to be improved. Summary of the invention

[0003] The present application provides a training method, system and device for a weather landscape recognition model, which solves the technical problem that the weather landscape recognition model may have misjudgment results during recognition due to classification errors in a sample image data set, thereby achieving the technical effect of increasing the model training speed and improving the accuracy of weather landscape recognition.

[0004] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0005] In a first aspect, an embodiment of the present application provides a method for training a weather landscape recognition model, the method comprising:

[0006] The initial recognition model is preliminarily trained using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images use the weather landscape labels;

[0007] Acquire a reference interference image, and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label;

[0008] The reference interference image is spliced ​​with the sample image to obtain an image to be matched, feature matching is performed using the coordinate information and a first feature map of the image to be matched at a first granularity to obtain a first granularity matching result, and feature matching is performed using the first granularity matching result, the coordinate information, and a second feature map of the image to be matched at a second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity;

[0009] If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, setting the interference label for the similar landscape image;

[0010] The intermediate recognition model is fine-tuned using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

[0011] The training method of the weather landscape recognition model proposed in the embodiment of the present application first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0012] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0013] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0014] Optionally, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the step of fine-tuning the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model includes:

[0015] The parameters of the feature extraction part are frozen, and the intermediate recognition model is trained using the reference interference image and the similar landscape image to adjust the parameters of the feature sorting and classification part to obtain the weather landscape recognition model.

[0016] Optionally, the initial recognition model includes an initial feature extraction part, a feature attention part and an initial feature sorting and classification part connected in sequence; the initial recognition model is preliminarily trained using the sample image data set to obtain an intermediate recognition model, including:

[0017] Using the initial feature extraction part to extract features from the sample images in the sample image data set to obtain sample initial features;

[0018] Using the feature attention part to enhance the main features of the sample initial features to obtain sample enhanced features;

[0019] Using the initial feature sorting and classification part to perform feature fusion on the sample enhancement features to obtain sample fusion features;

[0020] According to the loss value of the sample fusion feature, the parameters of the initial feature extraction part and the initial feature sorting and classification part are adjusted until the stopping condition is met to obtain the intermediate recognition model.

[0021] Optionally, the stop condition for performing preliminary training on the initial recognition model includes a first training stop condition and a second training stop condition, wherein:

[0022] The first training stop condition is that the accuracy of the initial recognition model in recognizing the test data set is the same in a set number of consecutive iterations;

[0023] The second training stop condition is that the number of times the sample image data set is traversed reaches a stop threshold;

[0024] The stop condition for fine-tuning the intermediate recognition model includes a first fine-tuning stop condition and a second fine-tuning stop condition, wherein:

[0025] The first fine-tuning stopping condition is that the accuracy of the intermediate recognition model in recognizing the test data set is the same in a set number of consecutive iterations;

[0026] The second fine-tuning stopping condition is that the number of times of traversing the reference interference image and the similar landscape image reaches a stopping threshold.

[0027] Optionally, the first granularity matching result includes a first matching point, where the first matching point is a pixel point in the image to be matched that has a matching relationship with a feature of the reference feature point; the second granularity matching result is obtained by the following method:

[0028] Performing image feature extraction on the image to be matched at the second granularity to obtain the second feature map;

[0029] Extracting the pixel points corresponding to the reference feature points in the second feature map to obtain a first reference feature map; extracting the pixel points corresponding to the first matching points in the second feature map to obtain a first sample feature map; splicing the first reference feature map with the first sample feature map to obtain a feature point matching image;

[0030] Feature matching is performed based on the coordinate information of the feature point matching image and the reference feature point to obtain the second granularity matching result; wherein the second granularity matching result includes a second matching point, and the second matching point is a pixel point in the feature point matching image that has a matching relationship with the feature of the reference feature point.

[0031] Optionally, the first granularity matching result is obtained by the following method:

[0032] Extracting image features of the image to be matched at the first granularity to obtain the first feature map;

[0033] Feature matching is performed on the image features of the sample image in the first feature map according to the coordinate information of the reference feature point to obtain the first matching point.

[0034] Optionally, whether the sample image in the to-be-matched image is the similar landscape image is determined in the following manner:

[0035] If the number of the second matching points exceeds a preset threshold, it is considered that the sample image and the reference interference image have the same reference interference item, and the sample image is determined to be the similar landscape image.

[0036] Optionally, the sample image dataset is obtained by the following method:

[0037] Preprocessing the collected original sample images, and obtaining the sample image data set according to the preprocessed original sample images; wherein the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement;

[0038] The color space enhancement includes:

[0039] Obtaining a weather landscape color mask according to the color data of the original sample image and the weather landscape color range; wherein the weather landscape color range is the color range corresponding to the weather landscape in the original sample image;

[0040] Performing color enhancement processing on the original sample image using the weather landscape color mask to obtain a color enhanced image;

[0041] The color enhanced image is smoothed to obtain the original sample image after color space enhancement preprocessing.

[0042] In a second aspect, an embodiment of the present application provides a training system for a weather landscape recognition model, the system comprising:

[0043] A preliminary training module, used to perform preliminary training on the initial recognition model using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images use the weather landscape labels;

[0044] An interference reference module, used to obtain a reference interference image and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label;

[0045] An image matching module is used to splice the reference interference image with the sample image to obtain an image to be matched, perform feature matching using the coordinate information and a first feature map of the image to be matched at a first granularity to obtain a first granularity matching result, and perform feature matching using the first granularity matching result, the coordinate information, and a second feature map of the image to be matched at a second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity;

[0046] an interference determination module, configured to set the interference label, and if it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, set the interference label for the similar landscape image;

[0047] The fine-tuning training module is used to fine-tune the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

[0048] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the method described in any one of the above embodiments by executing the computer instructions.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute any one of the methods in the above embodiments.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute any one of the methods in the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A step diagram of a method for training a weather landscape recognition model provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a weather landscape image and a sample interference image in an embodiment of the present application;

[0054] Figure 3 A diagram showing the steps of fine-tuning the intermediate recognition model in an embodiment of the present application;

[0055] Figure 4 A diagram showing the steps of training the initial recognition model in an embodiment of the present application;

[0056] Figure 5 A diagram showing the steps of performing feature matching on the second feature map in an embodiment of the present application;

[0057] Figure 6 This is a step diagram for performing feature matching on the first feature map in an embodiment of the present application;

[0058] Figure 7 A diagram showing the steps of color space enhancement in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of changes in the loss value and accuracy of the recognition model during the training process of an embodiment of the present application;

[0060] Fig. 9 A module diagram of a training system for a weather landscape recognition model provided in an embodiment of the present application;

[0061] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0063] Different regions have different geographical and climatic conditions, which means that different regions have different weather landscape resources. The conditions for the emergence of weather landscapes are very demanding, and usually require the coordination of multiple environmental conditions including atmospheric conditions, sunlight conditions, weather conditions, terrain conditions, and air pollution conditions, which makes it difficult to accurately predict the emergence and disappearance of weather landscapes. In addition, the duration of weather landscapes is also closely related to environmental conditions. Some weather landscapes can only last for a short time, which puts certain requirements on the speed of weather landscape prediction.

[0064] In related technologies, deep learning-based convolutional neural networks are usually used to learn features of sample images containing weather landscapes to obtain recognition models for identifying weather landscapes. In actual scenarios, sample images are usually collected by different people using different devices, resulting in a lack of consistency in the source, shooting angle, and clarity of the sample images. In addition, the recognition and classification of sample images are usually done manually. For weather landscapes with a short duration, classification errors are prone to occur when classifying sample images in a short period of time, which in turn affects the recognition accuracy of the recognition model.

[0065] Based on the above problems, the present application proposes a training method, system and equipment for a weather landscape recognition model. The method includes: using a sample image data set to perform preliminary training on an initial recognition model to obtain an intermediate recognition model; performing feature matching on the sample image based on a reference interference image to determine similar landscape images in the sample image data set; and using the reference interference image and similar landscape images to fine-tune the intermediate recognition model to obtain a weather landscape recognition model.

[0066] Among them, performing feature matching on the sample image based on the reference interference image includes: splicing the reference interference image with the sample image to obtain the image to be matched; performing feature matching using the coordinate information of the reference feature point and the first feature map of the image to be matched at the first granularity to obtain the first granularity matching result; performing feature matching using the first granularity matching result, the coordinate information and the second feature map of the image to be matched at the second granularity to obtain the second granularity matching result. Determining similar landscape images in the sample image data set includes: if the sample image in the image to be matched is determined to be a similar landscape image according to the second granularity matching result, setting the same interference label as the reference interference image for the similar landscape image.

[0067] The training method of the weather landscape recognition model proposed in the embodiment of the present application first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0068] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0069] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0070] The training method of the weather landscape recognition model provided in this specification can be applied to the training of neural network models, target detection models, transfer learning models and multimodal learning models, etc., to identify different weather landscapes. It should be noted that in a training process, the weather landscapes targeted are from the same region; and for weather landscapes in different regions, the corresponding sample image data sets can be used to train the recognition model. It can be understood that the training object can also be a machine learning algorithm, including a supervised learning algorithm and a deep learning algorithm, etc. After adaptively modifying this method, this method can also be used to identify other target objects besides weather landscapes.

[0071] According to an embodiment of the present application, an embodiment of a method for training a weather landscape recognition model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0072] In this embodiment, a method for training a weather landscape recognition model is provided, which can be used to train the above-mentioned image recognition model to recognize weather landscapes. Figure 1 As shown, the method includes:

[0073] S100. Perform preliminary training on the initial recognition model using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of similar landscape images use weather landscape labels.

[0074] S200. Acquire a reference interference image, and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label.

[0075] S300. Splice the reference interference image with the sample image to obtain the image to be matched, perform feature matching using the coordinate information and the first feature map of the image to be matched at the first granularity to obtain a first granularity matching result, perform feature matching using the first granularity matching result, the coordinate information and the second feature map of the image to be matched at the second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity.

[0076] S400: If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is a similar landscape image, an interference label is set for the similar landscape image.

[0077] S500. Fine-tune the intermediate recognition model using the reference interference image and the similar landscape image to obtain a weather landscape recognition model.

[0078] Specifically, the sample image data set may be a set of images collected in advance for the sky of any region, including weather landscape images or similar landscape images. Among them, the weather landscape image may be an image of the weather landscape of any region, and the weather landscape image may contain at least one weather landscape. The weather landscape image has a weather landscape label, and the weather landscape label may also be divided into multiple types according to the type of weather landscape contained in the weather landscape image, such as a blue sky landscape label and a white cloud landscape label. The similar landscape image may be an image collected at the same time as the weather landscape image, which contains sample interference items, and the sample interference items are the non-weather landscape parts of the sample image that have similar features to the weather landscape. When the classification time of the sample image data set is short or the classification accuracy is not high, the sample image cannot be accurately classified, and the similar landscape image and the weather landscape image will have the same weather landscape label in the sample image data set. In the process of model training, the sample interference items in the similar landscape image will cause the recognition model to learn features unrelated to the weather landscape, resulting in misjudgment results of the weather landscape recognition model during the recognition process.

[0079] Taking similar landscape images as an example, when the sample image is collected in the early morning or evening, the sample image may contain street lamps, which have the characteristics of high brightness and high contrast, and are similar to the weather landscape to be identified. At this time, when the classification accuracy is low, the sample image containing street lamps is regarded as a similar landscape image, and has the same weather landscape label as the weather landscape image. During the training process, the recognition model learns similar landscape images containing street lamps, and then misjudges the characteristics of street lamps as the characteristics of weather landscapes, causing the weather landscape recognition model to misjudge street lamps as weather landscapes during the recognition process.

[0080] It should be noted that the sample image dataset also includes sample interference images, which may be images that are classified and determined to contain sample interference items after the sample images are collected, and have interference labels. Figure 2 , Figure 2 The schematic diagram of the weather landscape image and the sample interference image in the embodiment of the present application is shown in the figure. As shown in the figure, in some embodiments, the weather landscape image can be obtained by taking a panoramic shot of the sky at the same angle at multiple locations in any area, including at least one of the weather landscapes such as blue sky, white clouds, morning glow and evening glow. The sample interference image can be an image collected at the same time as the weather landscape image, including at least one of the interference items such as night, cloudy day and rainy day.

[0081] Specifically, the reference interference image may be an image containing any type of reference interference item and having an interference label, wherein the reference interference item is an interference item of a determined type and is provided with a plurality of reference feature points. The reference feature point may be a pixel point occupied by the reference interference item in the image, and has features capable of characterizing the reference interference item, such as color, brightness or shape, etc., and the type and position of the reference interference item may be determined based on the features of the reference feature point. It is understandable that there may be multiple reference interference images for any type of reference interference item, thereby characterizing the features of the reference interference item in different aspects.

[0082] Specifically, the image to be matched can be obtained by splicing the reference interference image and the sample image. In order to determine whether the sample image contains the reference interference item in the reference interference image, feature matching is performed based on the features of the image to be matched at the first granularity and the features at the second granularity, wherein the first granularity is larger than the second granularity, the larger the granularity, the lower the resolution of the image, and the smaller the granularity, the higher the resolution of the image. Therefore, the first granularity matching result can be the feature matching result of the reference interference image and the sample image at a lower resolution, and the second granularity matching result can be the feature matching result of the reference interference image and the sample image at a higher resolution, and the accuracy of the second granularity matching result is greater than the first granularity matching result. Both the first granularity matching result and the second granularity matching result characterize the similarity between the sample image and the reference interference image, and as the granularity decreases, the accuracy of the feature matching result gradually increases. In the feature matching process, the feature matching result at each granularity is used as the basis for feature matching at the next granularity, thereby improving the efficiency of feature matching.

[0083] Correspondingly, when the accuracy requirement for the feature matching results is higher, a third granularity can also be set, and a third granularity matching result can be obtained based on the second granularity matching result, where the third granularity is smaller than the second granularity, and the resolution of the third feature map obtained based on the third granularity is higher than that of the second feature map.

[0084] When the second granularity feature matching can meet the accuracy requirements, if the sample image is determined to contain a reference interference item according to the second granularity feature matching result, the sample image is determined to be a similar landscape image, and an interference label is set for the similar landscape image. By performing feature matching on all sample images in the sample image dataset, the sample image dataset is further classified, the accuracy of the weather landscape features in the sample image dataset is improved, and the accuracy of the trained weather landscape recognition model is improved.

[0085] In some embodiments, after determining similar landscape images in a sample image dataset, reference interference items of the same type are determined based on the types of sample interference items in the similar landscape images, and reference interference images containing the above reference interference items are added to the sample image dataset to increase the number of features of the sample interference items and enhance the learning ability of the intermediate recognition model.

[0086] The training method of the weather landscape recognition model provided in this embodiment first uses the sample image data set to perform preliminary training on the initial recognition model, so that the initial recognition model can learn the global features of the sample image data set, provide a feature basis for subsequent training, and thus improve the speed of model training. Secondly, the intermediate recognition model is fine-tuned using reference interference images and similar landscape images, so that the intermediate recognition model can perform targeted parameter adjustments according to the features of the interference items, thereby reducing the probability of misjudgment of the weather landscape recognition model and improving the accuracy of weather landscape recognition.

[0087] This method can also automatically match the features of the sample images, search for similar landscape images in the sample image dataset by referring to the interference images, and set interference labels for similar landscape images to further classify the sample image dataset. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image dataset, optimizes the fine-tuning effect of the intermediate recognition model, and thus improves the accuracy of the weather landscape recognition model.

[0088] In addition, during the feature matching process of sample images, this method uses the features of the image to be matched at multiple granularities for progressive search, which effectively shortens the time required to determine the target matching points and increases the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0089] Reference Figure 3 As shown, as an embodiment of the present application, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the intermediate recognition model is fine-tuned using a reference interference image and a similar landscape image to obtain a weather landscape recognition model, including:

[0090] S510. Freeze the parameters of the feature extraction part, use the reference interference image and similar landscape images to train the intermediate recognition model, adjust the parameters of the feature sorting and classification part, and obtain the weather landscape recognition model.

[0091] Specifically, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part, wherein the feature extraction part is used to extract features from the sample image, and the feature sorting and classification part sorts and classifies the extracted features and learns the features. It should be noted that when fine-tuning the intermediate recognition model, the reference interference items in the reference interference image and the sample interference items in the similar landscape image are interference items of the same type and have the same features.

[0092] By freezing the parameters of the feature extraction part of the intermediate recognition model, the feature extraction ability of the intermediate recognition model is retained, so that the intermediate recognition model can learn according to the features of interference items in reference interference images and similar landscape images, and adjust the parameters of the feature sorting and classification part in a targeted manner, thereby obtaining a weather landscape recognition model, which effectively improves the efficiency of model training.

[0093] Reference Figure 4 As shown, as an embodiment of the present application, the initial recognition model includes an initial feature extraction part, a feature attention part and an initial feature sorting and classification part connected in sequence; the initial recognition model is preliminarily trained using a sample image data set to obtain an intermediate recognition model, including:

[0094] S110. Utilize the initial feature extraction part to extract features from the sample images in the sample image data set to obtain sample initial features.

[0095] S120. Use the feature attention part to enhance the main features of the sample initial features to obtain the sample enhanced features.

[0096] S130. Use the initial feature sorting and classification part to perform feature fusion on the sample enhancement features to obtain sample fusion features.

[0097] S140. According to the loss value of the sample fusion feature, the parameters of the initial feature extraction part and the initial feature sorting and classification part are adjusted until the stopping condition is met to obtain an intermediate recognition model.

[0098] Specifically, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part, which are connected in sequence. The initial feature extraction part is used to extract features from the sample image to obtain the sample initial features; the feature attention part is used to analyze the sample initial features and enhance the subject features of the part representing the main body of the image to obtain the sample enhanced features; the initial feature sorting and classification part is used to integrate according to the sample enhanced features to obtain the sample fusion features.

[0099] Furthermore, the feature attention part adopts the attention mechanism to determine the location of the image subject in the sample image according to the initial features of the sample, and enhances the features corresponding to the image subject, thereby highlighting the image subject in the sample image and improving the learning ability of the initial recognition model.

[0100] In some embodiments, the initial recognition model can be a ResNet-50 model, which is a residual convolutional neural network model that adds residual blocks to the traditional convolutional neural network, and directly transmits any layer of data to the subsequent layer through the residual block, thereby overcoming the problems of low learning efficiency and inability to improve model recognition accuracy due to increased network depth. The ResNet-50 model includes 49 convolutional layers and 1 fully connected layer, wherein the convolutional layer is used to extract features from the input sample image, such as edges, colors, and textures; the fully connected layer is connected to the convolutional layer to integrate the features extracted by the convolutional layer to perform tasks such as classification or regression.

[0101] Furthermore, after the sample image is input into the ResNet-50 model, the sample image first passes through a 7×7 convolution layer with a stride of 2 and a maximum pooling layer. The maximum pooling layer segments and reduces the dimension of the features extracted by the convolution layer to obtain a pooled feature map. Secondly, the pooled feature map is extracted through 16 residual blocks connected in sequence to obtain a residual feature map, and the residual feature map is converted into a fixed-size feature vector through a global average pooling layer. Finally, the feature vector is mapped to the label of the sample image through a fully connected layer, so as to predict and identify the type of the image subject in the sample image.

[0102] Furthermore, according to the difference between the output recognition result and the label of the sample image, the loss value is calculated, and the parameters in the initial feature extraction part and the initial feature sorting and classification part are adjusted according to the loss value until the stopping condition is met, thereby obtaining an intermediate recognition model.

[0103] As an embodiment of the present application, the stop condition for preliminary training of the initial recognition model includes a first training stop condition and a second training stop condition, wherein:

[0104] The first training stop condition is that the accuracy of the initial recognition model in recognizing the test data set is the same in a set number of consecutive iterations.

[0105] The second training stop condition is that the number of times the sample image data set is traversed reaches a stop threshold.

[0106] The stop condition for fine-tuning the intermediate recognition model includes a first fine-tuning stop condition and a second fine-tuning stop condition, wherein:

[0107] The first fine-tuning stopping condition is that the accuracy of the intermediate recognition model in recognizing the test data set is the same in a set number of consecutive iterations.

[0108] The second fine-tuning stopping condition is that the number of traversals of the reference interference image and the similar landscape image reaches a stopping threshold.

[0109] Specifically, the test data set may be collected simultaneously with the sample image data set, and include multiple test images. The image subjects in the test images are identified and determined in advance. The accuracy of the initial recognition model or the intermediate recognition model may be verified through the test data set.

[0110] Furthermore, in the process of training the initial recognition model, the accuracy of the initial recognition model gradually increases with the increase of the iteration rounds. If the accuracy of the initial recognition model remains the same in the iteration rounds of the set number of consecutive times, that is, the iterative training cannot improve the accuracy, then the preliminary training of the initial recognition model is stopped. Similarly, if in the process of fine-tuning the intermediate recognition model, the accuracy of the intermediate recognition model remains the same in the iteration rounds of the set number of consecutive times, then the fine-tuning of the intermediate recognition model is stopped.

[0111] Specifically, the performance of the intermediate recognition model is related to hyperparameters such as the learning rate strategy or training cycle during training. Different hyperparameter combinations correspond to different batch sizes. During training, the initial recognition model randomly selects sample images from the sample image dataset according to the batch size for learning until all sample images in the sample image dataset are traversed. If the number of traversals of the sample image dataset reaches the stopping threshold during training, it means that the initial recognition model has fully learned the sample image dataset. In order to avoid overfitting, the training of the initial recognition model can be stopped.

[0112] Similarly, in the process of fine-tuning the intermediate recognition model, the intermediate recognition model needs to learn the reference interference images and similar landscape images. If the number of traversals of the reference interference images and similar landscape images reaches the stop threshold, it means that the intermediate recognition model has fully learned the reference interference images and similar landscape image sets. In order to avoid overfitting, the fine-tuning of the intermediate recognition model can be stopped.

[0113] Reference Figure 5 As shown, as an embodiment of the present application, the first granularity matching result includes a first matching point, which is a pixel point in the image to be matched that has a matching relationship with the feature of the reference feature point; the second granularity matching result is obtained by the following method:

[0114] S310. Perform feature extraction on the image to be matched at a second granularity to obtain a second feature map.

[0115] S320. Take out the pixel points corresponding to the reference feature points in the second feature map to obtain a first reference feature map; take out the pixel points corresponding to the first matching point in the second feature map to obtain a first sample feature map; splice the first reference feature map and the first sample feature map to obtain a feature point matching image.

[0116] S330. Perform feature matching based on the coordinate information of the feature point matching image and the reference feature point to obtain a second granularity matching result; wherein the second granularity matching result includes a second matching point, which is a pixel point in the feature point matching image that has a matching relationship with the feature of the reference feature point.

[0117] Specifically, since the first granularity is larger than the second granularity, the resolution of the first feature map is lower, and it contains more comprehensive feature information, which can describe the global features of the reference interference image and the sample image; while the resolution of the second feature map is higher, and the features contained therein are more accurate, which can describe the detailed features of the reference interference image and the sample image. After pixel point extraction, the first reference feature map only contains features around the reference feature point, and the first sample feature map only contains features around the first matching point, thereby effectively removing features in the second feature map that are not related to the reference feature point and the first matching point, reducing the amount of calculation required for feature matching, and then being able to determine the second matching point more quickly, improving the efficiency of feature matching.

[0118] This method uses the features of the image to be matched at multiple granularities to perform a progressive search, effectively shortening the time required to determine the target matching point and increasing the speed of searching for similar landscape images through reference interference images, thereby shortening the time to construct a training data set for fine-tuning and improving the training speed of the weather landscape recognition model.

[0119] It is understandable that the first reference feature map can be taken from the reference interference image, the first sample feature map can be taken from the sample image, and the first reference feature map and the first sample feature map are spliced ​​to obtain a feature point matching image. The first reference feature map and the first sample feature map can also be directly taken from the second feature map to obtain a feature point matching image.

[0120] Furthermore, based on the second granularity matching result, similar landscape images in the sample image dataset can be searched, and interference labels can be set for similar landscape images to distinguish between similar landscape images and weather landscape images. Compared with the related art of manually selecting sample images to construct a training dataset for fine-tuning, this method uses reference interference images to automatically search for similar landscape images in the sample image dataset, thereby improving the accuracy of the sample images in the sample image dataset, thereby improving the accuracy of the weather landscape recognition model.

[0121] Reference Figure 6 As shown, as an embodiment of the present application, the first granularity matching result is obtained by the following method:

[0122] S340. Perform image feature extraction on the image to be matched at a first granularity to obtain a first feature map.

[0123] S350. Perform feature matching on the image features of the sample image in the first feature map according to the coordinate information of the reference feature point to obtain a first matching point.

[0124] Specifically, in the feature matching process, the position of the reference feature point in the first feature map is determined according to the coordinate information of the reference feature point, and the image features of the pixel points at the corresponding position are extracted. According to the above image features, matching is performed in the part of the first feature map related to the sample image, so as to determine the pixel points having the above matching image features to obtain the first matching point.

[0125] As an embodiment of the present application, whether a sample image in the image to be matched is a similar landscape image is determined in the following manner:

[0126] S360. If the number of the second matching points exceeds the preset threshold, it is considered that the sample image and the reference interference image have the same reference interference item, and the sample image is determined to be a similar landscape image.

[0127] Specifically, if the number of second matching points exceeds a preset threshold, it means that there are multiple pixel points in the sample image that are similar to the features of the reference interference items. According to the number of second matching points, it can be determined that the sample image and the reference interference item have the same reference interference item, and the sample image is judged as a similar landscape image. Similar landscape images in the sample image data set will cause the initial recognition model to learn incorrect features, resulting in misjudgment results of the weather landscape recognition model during the recognition process. This method uses reference interference images to automatically search for similar landscape images, improves the accuracy of sample images in the sample image data set, optimizes the fine-tuning effect of the intermediate recognition model, and thereby improves the accuracy of the weather landscape recognition model.

[0128] In some embodiments, after determining similar landscape pictures, reference interference pictures containing reference interference items of the same type are determined based on the types of sample interference items therein, and these reference interference pictures are added to the sample image data set as sample pictures, thereby increasing the number of interference item features and improving the effect of fine-tuning the intermediate recognition model, thereby improving the accuracy of the weather landscape recognition model.

[0129] Reference Figure 7 As shown, as an embodiment of the present application, a sample image data set is obtained by the following method:

[0130] S610. Preprocess the collected original sample images, and obtain a sample image data set according to the preprocessed original sample images; wherein the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement.

[0131] Color space enhancements include:

[0132] S620. Obtain a weather landscape color mask according to the color data of the original sample image and the weather landscape color range; wherein the weather landscape color range is the color range corresponding to the weather landscape in the original sample image.

[0133] S630. Perform color enhancement processing on the original sample image using the weather landscape color mask to obtain a color enhanced image.

[0134] S640. Perform smoothing on the color enhanced image to obtain an original sample image after color space enhancement preprocessing.

[0135] Specifically, the preprocessing includes at least one of normalization, resizing, contrast adjustment, and color space enhancement. Among them, normalization can adjust the features of the sample images to the same range to improve the stability of model training; resizing can adjust the sample images to the same size to reduce the interference of image size model training; contrast adjustment can enhance the contrast difference between the image subject and the background in the sample image, thereby highlighting the image subject.

[0136] Furthermore, color space enhancement can be used to enhance the color of the main body of the image in the sample image, thereby strengthening the characteristics of the main body of the image. In color space enhancement, a weather landscape color mask is first determined based on the color data of the original sample image and the weather landscape color range, wherein the weather landscape color range is determined in advance based on the weather landscape to be identified. Based on the weather landscape color range, the pixel points in the original sample image whose colors are within the weather landscape color range are used as the weather landscape, thereby obtaining a weather landscape color mask about the weather landscape.

[0137] Secondly, the weather landscape color mask is used to perform color enhancement processing on the weather landscape in the original sample image to obtain a color enhanced image. Specifically, the color enhancement processing includes weighting the color of the weather landscape to increase the weight of the corresponding color of the weather landscape, thereby highlighting the color of the weather landscape in the original sample image.

[0138] Finally, the color enhanced image is smoothed to reduce the color discontinuity after color enhancement, and the brightness and contrast of the color enhanced image are further adjusted to obtain the original sample image after color space enhancement preprocessing. By performing color space enhancement on the original sample image, the weather landscape as the main body of the image in the original sample image is made more prominent, thereby improving the learning ability of the initial recognition model.

[0139] The following introduces an actual scenario of a training method for a weather landscape recognition model provided by the present application. The sample image data used in this actual scenario is a historical observation picture obtained by a weather phenomenon video intelligent observation instrument for multiple national site locations in the Beijing area. Among them, the weather phenomenon video intelligent observation instrument uses a fixed fisheye lens to capture a circular area within a 360-degree range of the sky. Multiple national sites include Shunyi, Haidian, Yanqing, Foyeding, Tanghekou, Huairou, Miyun, Shangdianzi, Pinggu, Tongzhou, Chaoyang, Changping, Zhaitang, Mentougou, Observatory, Shijingshan, Fengtai, Daxing, Fangshan and Xiayunling. The time range is from August 29, 2020 to November 30, 2021. The sample image collection interval is 30 minutes, and the weather landscape in the sample image includes blue sky, white clouds, morning glow and sunset glow. Based on the above sample images, a sample image dataset is obtained after preprocessing, and a weather landscape recognition model is obtained by training and fine-tuning the ResNet-50 model.

[0140] Reference Figure 8 , Figure 8 The present invention is a schematic diagram of the change of the loss value and accuracy of the recognition model during the training process of the embodiment of the present invention. As shown in the figure, during the training process, the training loss value (Train Loss) of the weather landscape recognition model gradually decreases with the increase of iteration rounds, and the training accuracy (Train Accuracy) gradually increases with the increase of iteration rounds. The final weather landscape recognition model has an accuracy of 91.72% for recognizing the sample image data set. The weather landscape recognition model obtained in each iteration round is tested, and the test loss value (Test Loss) of the weather landscape recognition model is stable at 0.4, and the test accuracy (Test Accuracy) of the weather landscape recognition model for recognizing the test data set is 89.97%.

[0141] The accuracy of the weather landscape recognition model for identifying different types of weather landscapes is shown in Table 1. Among them, the accuracy of the weather landscape recognition model for identifying blue sky and white clouds can reach more than 90%, the accuracy of identifying sunset glow is close to 90%, and the accuracy of identifying morning glow is slightly lower, close to 80%.

[0142] Table 1 The accuracy of weather landscape recognition model in identifying different types of weather landscapes

[0143] Blue sky interference Morning glow Sunset White Cloud average Sample image dataset 96.97％ 91.16％ 87.33％ 94.37％ 87.27％ 91.72％ Test Dataset 95.43％ 90.11％ 79.45％ 89.94％ 92.00％ 89.97％

[0144] Accordingly, please refer to Fig. 9 , an embodiment of the present application provides a training system for a weather landscape recognition model, the system comprising:

[0145] The preliminary training module 100 is used to perform preliminary training on the initial recognition model using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of similar landscape images use weather landscape labels.

[0146] The interference reference module 200 is used to obtain a reference interference image and determine the coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label.

[0147] The image matching module 300 is used to splice the reference interference image with the sample image to obtain the image to be matched, perform feature matching using the coordinate information and the first feature map of the image to be matched at the first granularity to obtain a first granularity matching result, and perform feature matching using the first granularity matching result, the coordinate information and the second feature map of the image to be matched at the second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity.

[0148] The interference determination module 400 is used to set an interference tag. If the sample image in the to-be-matched image is determined to be a similar landscape image according to the second granularity matching result, an interference tag is set for the similar landscape image.

[0149] The fine-tuning training module 500 is used to fine-tune the intermediate recognition model using reference interference images and similar landscape images to obtain a weather landscape recognition model.

[0150] In some optional implementations, the intermediate recognition model includes a feature extraction part and a feature sorting and classification part; the fine-tuning training module 500 includes:

[0151] The freezing fine-tuning unit is used to freeze the parameters of the feature extraction part, and use the reference interference image and similar landscape images to train the intermediate recognition model to adjust the parameters of the feature sorting and classification part to obtain the weather landscape recognition model.

[0152] In some optional embodiments, the initial recognition model includes an initial feature extraction part, a feature attention part, and an initial feature sorting and classification part connected in sequence; the preliminary training module 100 includes:

[0153] The initial feature extraction unit is used to extract features from the sample images in the sample image data set using the initial feature extraction part to obtain the sample initial features.

[0154] The initial feature enhancement unit is used to enhance the main features of the sample initial features by using the feature attention part to obtain the sample enhanced features.

[0155] The initial feature fusion unit is used to use the initial feature sorting and classification part to fuse the sample enhancement features to obtain the sample fusion features.

[0156] The initial parameter adjustment unit is used to adjust the parameters of the initial feature extraction part and the initial feature sorting and classification part according to the loss value of the sample fusion feature until the stopping condition is met to obtain an intermediate recognition model.

[0157] In some optional implementations, the stop condition in the preliminary training module 100 includes a first training stop condition and a second training stop condition, wherein:

[0158] The first training stop condition is that the accuracy of the initial recognition model in recognizing the test data set is the same in a set number of consecutive iterations.

[0159] The second training stop condition is that the number of times the sample image data set is traversed reaches a stop threshold.

[0160] The stop condition in the fine-tuning training module 500 includes a first fine-tuning stop condition and a second fine-tuning stop condition, wherein:

[0161] The first fine-tuning stopping condition is that the accuracy of the intermediate recognition model in recognizing the test data set is the same in a set number of consecutive iterations.

[0162] The second fine-tuning stopping condition is that the number of traversals of the reference interference image and the similar landscape image reaches a stopping threshold.

[0163] In some optional implementations, the first granularity matching result includes a first matching point, which is a pixel point in the image to be matched that has a matching relationship with a feature of the reference feature point; the image matching module 300 includes:

[0164] The second feature extraction unit is used to extract image features of the image to be matched at a second granularity to obtain a second feature map.

[0165] The feature extraction unit is used to extract the pixel points corresponding to the reference feature points in the second feature map to obtain the first reference feature map; extract the pixel points corresponding to the first matching point in the second feature map to obtain the first sample feature map; and splice the first reference feature map with the first sample feature map to obtain a feature point matching image.

[0166] The second granularity matching unit is used to perform feature matching based on the coordinate information of the feature point matching image and the reference feature point to obtain a second granularity matching result; wherein the second granularity matching result includes a second matching point, which is a pixel point in the feature point matching image that has a matching relationship with the feature of the reference feature point.

[0167] In some optional implementations, the image matching module 300 further includes:

[0168] The first feature extraction unit is used to extract image features of the image to be matched at a first granularity to obtain a first feature map.

[0169] The first granularity matching unit is used to perform feature matching on the image features of the sample image in the first feature map according to the coordinate information of the reference feature point to obtain a first matching point.

[0170] In some optional implementations, the image matching module 300 further includes:

[0171] The similar landscape image judging unit is used to judge whether the sample image is a similar landscape image. If the number of second matching points exceeds a preset threshold, it is considered that the sample image and the reference interference image have the same reference interference item, and the sample image is judged to be a similar landscape image.

[0172] In some optional embodiments, the system further includes a sample acquisition module, including:

[0173] The sample preprocessing unit is used to preprocess the collected original sample images and obtain a sample image data set based on the preprocessed original sample images; wherein the preprocessing includes at least one of normalization, size adjustment, contrast adjustment and color space enhancement.

[0174] Wherein, the sample preprocessing unit includes:

[0175] The mask determination subunit is used to obtain a weather landscape color mask according to the color data of the original sample image and the weather landscape color range; wherein the weather landscape color range is the color range corresponding to the weather landscape in the original sample image.

[0176] The color enhancement subunit is used to perform color enhancement processing on the original sample image using the weather landscape color mask to obtain a color enhanced image.

[0177] The smoothing processing subunit is used to perform smoothing on the color enhanced image to obtain the original sample image after color space enhancement preprocessing.

[0178] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0179] The training system of the weather landscape recognition model in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0180] See also Fig.10 , Fig.10 It is a structural diagram of a computer device provided by an embodiment of the present application. As shown in the figure, the computer device includes: one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Each component uses different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0181] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0182] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0183] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0184] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0185] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0186] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0187] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0188] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

[0189] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0190] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0191] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0195] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0196] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0197] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

[0198] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for training a weather landscape recognition model, characterized in that: The method comprises: The initial recognition model is preliminarily trained using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images use the weather landscape labels; Acquire a reference interference image, and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label; The reference interference image is spliced ​​with the sample image to obtain an image to be matched, feature matching is performed using the coordinate information and a first feature map of the image to be matched at a first granularity to obtain a first granularity matching result, and feature matching is performed using the first granularity matching result, the coordinate information, and a second feature map of the image to be matched at a second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity; If it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, setting the interference label for the similar landscape image; The intermediate recognition model is fine-tuned using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

2. The method according to claim 1, characterized in that The intermediate recognition model includes a feature extraction part and a feature sorting and classification part; The step of fine-tuning the intermediate recognition model by using the reference interference image and the similar landscape image to obtain the weather landscape recognition model comprises: The parameters of the feature extraction part are frozen, and the intermediate recognition model is trained using the reference interference image and the similar landscape image to adjust the parameters of the feature sorting and classification part to obtain the weather landscape recognition model.

3. The method according to claim 1, characterized in that The initial recognition model includes an initial feature extraction part, a feature attention part and an initial feature sorting and classification part connected in sequence; The initial recognition model is preliminarily trained using the sample image data set to obtain an intermediate recognition model, including: Using the initial feature extraction part to extract features from the sample images in the sample image data set to obtain sample initial features; Using the feature attention part to enhance the main features of the sample initial features to obtain sample enhanced features; Using the initial feature sorting and classification part to perform feature fusion on the sample enhancement features to obtain sample fusion features; According to the loss value of the sample fusion feature, the parameters of the initial feature extraction part and the initial feature sorting and classification part are adjusted until the stopping condition is met to obtain the intermediate recognition model.

4. The method according to claim 3, characterized in that The stop condition for performing preliminary training on the initial recognition model includes a first training stop condition and a second training stop condition, wherein: The first training stop condition is that the accuracy of the initial recognition model in recognizing the test data set is the same in a set number of consecutive iterations; The second training stop condition is that the number of times the sample image data set is traversed reaches a stop threshold; The stop condition for fine-tuning the intermediate recognition model includes a first fine-tuning stop condition and a second fine-tuning stop condition, wherein: The first fine-tuning stopping condition is that the accuracy of the intermediate recognition model in recognizing the test data set is the same in a set number of consecutive iterations; The second fine-tuning stopping condition is that the number of times of traversing the reference interference image and the similar landscape image reaches a stopping threshold.

5. The method according to claim 1, characterized in that The first granularity matching result includes a first matching point, where the first matching point is a pixel point in the image to be matched that has a matching relationship with a feature of the reference feature point; The second granularity matching result is obtained by the following method: Performing image feature extraction on the image to be matched at the second granularity to obtain the second feature map; Extract the pixel points corresponding to the reference feature points in the second feature map to obtain a first reference feature map; Take out the pixel points corresponding to the first matching points in the second feature map to obtain a first sample feature map; splice the first reference feature map with the first sample feature map to obtain a feature point matching image; Feature matching is performed based on the coordinate information of the feature point matching image and the reference feature point to obtain the second granularity matching result; wherein the second granularity matching result includes a second matching point, and the second matching point is a pixel point in the feature point matching image that has a matching relationship with the feature of the reference feature point.

6. The method according to claim 5, characterized in that The first granularity matching result is obtained by the following method: Extracting image features of the image to be matched at the first granularity to obtain the first feature map; Feature matching is performed on the image features of the sample image in the first feature map according to the coordinate information of the reference feature point to obtain the first matching point.

7. The method according to claim 5, characterized in that Whether the sample image in the image to be matched is the similar landscape image is determined by: If the number of the second matching points exceeds a preset threshold, it is considered that the sample image and the reference interference image have the same reference interference item, and the sample image is determined to be the similar landscape image.

8. The method according to any one of claims 1 to 7, characterized in that The sample image dataset is obtained by the following method: Preprocessing the collected original sample images, and obtaining the sample image data set according to the preprocessed original sample images; wherein the preprocessing includes at least one of normalization, size adjustment, contrast adjustment, and color space enhancement; The color space enhancement includes: Obtaining a weather landscape color mask according to the color data of the original sample image and the weather landscape color range; wherein the weather landscape color range is the color range corresponding to the weather landscape in the original sample image; Performing color enhancement processing on the original sample image using the weather landscape color mask to obtain a color enhanced image; The color enhanced image is smoothed to obtain the original sample image after color space enhancement preprocessing.

9. A training system for a weather landscape recognition model, characterized in that: The system comprises: A preliminary training module, used to perform preliminary training on the initial recognition model using a sample image data set to obtain an intermediate recognition model; wherein the sample images in the sample image data set are weather landscape images or similar landscape images, the weather landscape images have weather landscape labels, and the labels of the similar landscape images use the weather landscape labels; An interference reference module, used to obtain a reference interference image and determine coordinate information of a pre-specified reference feature point in the reference interference image; wherein the reference interference image has an interference label; An image matching module is used to splice the reference interference image with the sample image to obtain an image to be matched, perform feature matching using the coordinate information and a first feature map of the image to be matched at a first granularity to obtain a first granularity matching result, and perform feature matching using the first granularity matching result, the coordinate information, and a second feature map of the image to be matched at a second granularity to obtain a second granularity matching result; wherein the first granularity is larger than the second granularity; an interference determination module, configured to set the interference label, and if it is determined according to the second granularity matching result that the sample image in the to-be-matched image is the similar landscape image, set the interference label for the similar landscape image; The fine-tuning training module is used to fine-tune the intermediate recognition model using the reference interference image and the similar landscape image to obtain the weather landscape recognition model.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected to communicate with each other. The memory stores computer instructions, and the processor executes the computer instructions. The method according to any one of claims 1 to 8 is thereby performed.

Images