A weather feature target inspection method, device, equipment, and medium

Abstract

The present invention relates to the field of image processing technology, and discloses a weather feature target inspection method, device, equipment, and medium. The method includes: obtaining continuously collected historical image data, and determining a corresponding first optical flow map based on the historical image data; determining a target detection data set based on the historical image data and the first optical flow map; training a target detection model based on the target detection data set to obtain a trained weather feature detection model; obtaining continuously collected image data to be inspected, and determining a corresponding second optical flow map based on the image data to be inspected; performing feature fusion using the weather feature detection model based on the image data to be inspected and the second optical flow map to determine a weather feature target result. The solution of the present invention uses the target detection data set determined by the image data and the optical flow map as the model training data, enabling the model to focus on regions with significant weather features, while reducing the computational resource requirements and ensuring the accuracy of the weather feature target result.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, device, equipment, and medium for weather feature target detection. Background Art

[0002] With the rapid development of computer vision technology, image classification has been widely applied in many fields. Especially in weather recognition, analyzing image data to judge weather conditions has become a research hotspot. Traditional weather recognition methods mainly rely on meteorological sensors and meteorological models. Although these methods are accurate, they are costly and have high requirements for environmental conditions.

[0003] In recent years, deep learning technology, especially convolutional neural network (CNN), has shown excellent performance in image classification tasks. Using CNN for image classification has achieved remarkable results in fields such as object recognition and face recognition. However, applying this technology to weather recognition still faces some challenges. For example, how to effectively handle the impact of various lighting and weather condition changes on image quality, and how to handle the diversity of weather features in images.

[0004] In related technologies, weather recognition methods mainly rely on multi-modal data fusion and complex feature extraction techniques. These methods are relatively complex to implement and have high requirements for computing resources. Therefore, there is a corresponding need for recognition methods in related technologies that have low requirements for computing resources and can accurately achieve weather prediction. Summary of the Invention

[0005] In view of this, the present invention provides a method, device, equipment, and medium for weather feature target detection to solve the technical problem of high computing resource requirements in related technologies.

[0006] In a first aspect, the present invention provides a method for weather feature target detection. The method includes: obtaining continuously collected historical image data, and determining a corresponding first optical flow map based on the historical image data; determining a target detection data set based on the historical image data and the first optical flow map; training a target detection model based on the target detection data set to obtain a trained weather feature detection model; obtaining continuously collected image data to be detected, and determining a corresponding second optical flow map based on the image data to be detected; performing feature fusion using the weather feature detection model based on the image data to be detected and the second optical flow map to determine a weather feature target result.

[0007] In combination with the first aspect, in a possible implementation manner of the first aspect, determining a target detection data set based on historical image data and a first optical flow map includes: inputting the historical image data into a pre-trained multi-modal model to determine an initial target detection data set; determining the positions of weather dynamic elements based on the first optical flow map; and determining the target detection data set based on the initial target detection data set and the positions of the weather dynamic elements.

[0008] In combination with the first aspect, in a possible implementation manner of the first aspect, inputting the historical image data into a pre-trained multi-modal model to determine an initial target detection data set includes: inputting the historical image data into a pre-trained multi-modal model to obtain a feature heat map and an image prediction classification result; based on the feature heat map, using a first target box to label the positions of corresponding high-heat regions and annotating the high-heat regions to form a first label; and generating an initial target detection data set based on the first target box and the first label.

[0009] In combination with the first aspect, in a possible implementation manner of the first aspect, determining the positions of weather dynamic elements based on the first optical flow map includes: using a second target box to label the weather dynamic elements based on the first optical flow map; determining the positions of the weather dynamic elements based on the second target box, and using the weather type corresponding to the weather dynamic elements as the second label of the second target box.

[0010] In combination with the first aspect, in a possible implementation manner of the first aspect, performing feature fusion using a weather feature detection model based on the image data to be detected and a second optical flow map to determine a weather feature target result includes: performing feature fusion using a weather feature detection model based on the image data to be detected and the second optical flow map to determine a weather feature prediction result and a confidence level corresponding to the weather feature prediction result; and screening the weather feature prediction results based on a preset confidence level threshold and the confidence level to determine the weather feature target result.

[0011] In combination with the first aspect, in a possible implementation manner of the first aspect, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder.

[0012] Based on the image data to be inspected and the second optical flow map, a weather feature detection model is used for feature fusion to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result, including: inputting the image data to be inspected at the first moment into the first image feature encoder to determine the first image feature; inputting the image data to be inspected at the second moment into the second image feature encoder to determine the second image feature; inputting the second optical flow map into the third image feature encoder to determine the third image feature; inputting the first image feature, the second image feature, and the third image feature into the feature fusion encoder to determine the fusion feature; based on the feature decoder, decoding the fusion feature to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result.

[0013] Combined with the first aspect, in a possible implementation manner of the first aspect, determining the corresponding first optical flow map based on the historical image data includes: determining the corresponding first optical flow map based on the historical image data at the first moment and the historical image data at the second moment by using the optical flow method.

[0014] In a second aspect, the present invention provides a weather feature target inspection device, including: a historical image acquisition module, configured to acquire continuously collected historical image data and determine the corresponding first optical flow map based on the historical image data; a data set determination module, configured to determine a target detection data set based on the historical image data and the first optical flow map; a model training module, configured to train a target detection model based on the target detection data set to obtain a trained weather feature detection model; an image to be inspected acquisition module, configured to acquire continuously collected image data to be inspected and determine the corresponding second optical flow map based on the image data to be inspected; a result determination module, configured to perform feature fusion by using the weather feature detection model based on the image data to be inspected and the second optical flow map to determine the weather feature target result.

[0015] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the weather feature target inspection method of the first aspect or any corresponding implementation manner thereof.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the weather feature target inspection method of the first aspect or any corresponding implementation manner thereof.

[0017] The technical solution of the present invention has the following advantages:

[0018] A method, device, equipment, and medium for weather feature target inspection provided by the present invention. This method determines a target detection data set through historical image data and a first optical flow map, and uses the target detection data set to complete the training of a target detection model. Then, the obtained image data and optical flow map are input into the trained weather feature detection model, and feature fusion is used to determine the weather feature target result. In this process, by using historical image data and the first optical flow map to determine the target detection data set, the data in the target detection data set has more obvious weather factor characteristics compared to the original data, ignoring other regions, thereby reducing the data calculation amount. On this basis, using the target detection data set as the training data for the model enables the model to learn and master the corresponding weather factor characteristics through training, so that the weather feature detection model pays more attention to the obvious feature regions. Therefore, when the image data and optical flow map are input into the weather feature detection model, the model only needs to focus on the regions with obvious features, and since the obvious features of these regions are related to weather factors, it is possible to ensure the accuracy of the weather feature target result while reducing the computational resource requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a flowchart showing a method for weather feature target inspection according to an embodiment of the present invention;

[0021] Figure 2 It is a flowchart showing the application of a weather feature detection model according to an embodiment of the present invention;

[0022] Figure 3 It is a structural block diagram of a weather feature target inspection device according to an embodiment of the present invention;

[0023] Figure 4 It is a schematic hardware structure diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] According to an embodiment of the present invention, an embodiment of a method for testing weather feature targets is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0026] This embodiment provides a method for testing weather feature targets, as Figure 1 shown, the method includes the following steps:

[0027] S101. Obtain continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data.

[0028] Specifically, determining a corresponding optical flow map based on continuously collected historical image data means using the previous-phase image data and the subsequent-phase image data, and using the optical flow method to determine the corresponding first optical flow map.

[0029] Specifically, the continuously collected historical image data refers to pre-collected image data that is continuous in time. For example, the historical image data at the first moment and the historical image data at the second moment are two image data that are continuous in collection time, that is, the previous-phase image data and the subsequent-phase image data.

[0030] S102. Determine a target detection data set based on the historical image data and the first optical flow map.

[0031] Specifically, determining a target detection data set based on the historical image data and the first optical flow map means respectively determining the regions with obvious weather features in the historical image data and the regions with weather dynamic elements in the first optical flow map, so as to jointly use the regions with obvious weather features and the regions with weather dynamic elements as the target detection data set. It should be understood that the regions with obvious weather features and the regions with weather dynamic elements are represented in the form of target boxes, and the corresponding positions are the positions of the corresponding regions in the image, and each corresponding region has a corresponding specific weather type. Among them, the historical image data refers to pre-collected image data, and the historical image data is used as training data for application.

[0032] S103. Train the object detection model based on the object detection dataset to obtain a trained weather feature detection model.

[0033] Specifically, training the object detection model based on the object detection dataset to obtain a trained weather feature detection model means using the object detection dataset as training data to train the object detection model until the object detection model converges. Generally, at this time, the loss function basically no longer decreases or the model reaches the maximum number of iterations, and it is considered that the object detection model is trained, thus obtaining the weather feature detection model.

[0034] S104. Obtain continuously acquired image data to be inspected and determine the corresponding second optical flow map based on the image data to be inspected.

[0035] Specifically, determining the corresponding second optical flow map based on the continuously acquired image data to be inspected means using the previous-phase image data and the subsequent-phase image data and using the optical flow method to determine the corresponding second optical flow map.

[0036] Specifically, the continuously acquired image data to be inspected refers to image data that is continuously acquired in real time in terms of time. For example, the image data to be inspected at the first moment and the image data to be inspected at the second moment are two image data that are continuous in terms of acquisition time, that is, the previous-phase image data and the subsequent-phase image data.

[0037] S105. Based on the image data to be inspected and the second optical flow map, use the weather feature detection model for feature fusion to determine the weather feature target result.

[0038] Specifically, the weather feature detection model has three image feature encoders, a feature fusion encoder, and a feature decoder. The image feature encoders are respectively used to extract the image features of the previous-phase image data, the subsequent-phase image data, and the optical flow map; the feature fusion encoder is used to fuse the extracted image features to obtain the fused image features; the feature decoder is used to obtain weather feature object detection frames of different sizes, that is, the weather feature target result.

[0039] The present invention provides a method, device, equipment, and medium for weather feature target inspection. The method determines a target detection data set through historical image data and a first optical flow map, and uses the target detection data set to complete the training of a target detection model. Then, the obtained image data and optical flow map are input into the trained weather feature detection model, and feature fusion is used to determine the weather feature target result. In this process, by using historical image data and a first optical flow map to determine the target detection data set, the data in the target detection data set has more obvious weather factor features compared with the original data, ignoring other regions, thereby reducing the data calculation amount. On this basis, using the target detection data set as the training data of the model, the model learns through training to master the corresponding weather factor features, so that the weather feature detection model pays more attention to the obvious feature regions. Therefore, when the image data and optical flow map are input into the weather feature detection model, the model only needs to pay attention to the regions with obvious features, and since the obvious features of these regions are related to weather factors, while reducing the computing resource requirements, the accuracy of the weather feature target result can be guaranteed.

[0040] In an alternative embodiment, determining the target detection data set based on the historical image data and the first optical flow map includes:

[0041] Inputting the historical image data into a pre-trained multi-modal model to determine an initial target detection data set; determining the positions of weather dynamic elements based on the first optical flow map; and determining the target detection data set based on the initial target detection data set and the positions of the weather dynamic elements.

[0042] Specifically, a pre-trained multi-modal model refers to a model that supports text and images, such as the CLIP (Contrastive Language-Image Pre-training) model. When the pre-trained multi-modal model outputs, it will output an image feature layer. The image feature layer determines the classification of the image through a fully connected layer, and through a branch convolutional layer and an upsampling layer, it obtains a feature heatmap with the same size as the input image of the model and the weather type classification corresponding to each feature heatmap. Among them, the high-heat regions in each feature heatmap and the corresponding weather type classifications are the initial target detection datasets. The manifestation form of the initial target detection datasets is the first target box and the weather type classification corresponding to the first target box. For example, in the historical image data collected by camera 1, the corresponding high-heat region may be (x1, y1), and in the historical image data collected by camera 2, the corresponding high-heat regions may be (x2, y1), (x3, y2), etc. It should be understood that each camera may have one or more high-heat regions. Among them, since each feature heatmap has a corresponding classification, and the position of the high-heat region in each feature heatmap is the position with high-dominant weather characteristics, therefore, this classification is related to the position of the high-heat region in each feature heatmap, thus forming a corresponding relationship between the camera and the high-sensitive region. Taking the high-heat regions corresponding to camera 1 as (x1, y1) and (x2, y2) as an example, (x1, y1) corresponding to camera 1 may be a position with high-dominant weather characteristics relative to cloudy days, and (x2, y2) corresponding to camera 1 may be a position with high-dominant weather characteristics relative to sunny days.

[0043] Specifically, determining the position of weather dynamic elements based on the first optical flow map means using the characteristic of the optical flow map to display dynamic weather changes, using the second target box to label the regional position with this characteristic, and labeling the weather type corresponding to this characteristic. For example: The weather type corresponding to the weather characteristic corresponding to the position labeled by the second target box is wind. Wind will cause the camera to shake, and the change characteristic of this type of wind will be shown in the optical flow map. The second target box will label the region corresponding to this change characteristic, such as (x3, y3); The weather type corresponding to the weather characteristic corresponding to the position labeled by the second target box is rain. Rain will have many diagonal lines, and the change characteristic of this type of rain will be shown in the optical flow map. The second target box will label the region corresponding to this change characteristic, such as (x4, y3).

[0044] Specifically, determining the target detection dataset based on the initial target detection dataset and the position of weather dynamic elements means taking the above-mentioned first target box, second target box, and their respective corresponding weather types as the target detection dataset, so as to jointly use the region with dominant weather characteristics and the region with weather dynamic elements as the target detection dataset.

[0045] In an alternative embodiment, historical image data is input into a pre-trained multi-modal model to determine an initial object detection dataset, including:

[0046] Input the historical image data into the pre-trained multi-modal model to obtain a feature heatmap and an image prediction classification result; based on the feature heatmap, use a first bounding box to label the positions of corresponding high-heat regions, and label the high-heat regions to form a first label; based on the first bounding box and the first label, generate an initial object detection dataset.

[0047] Specifically, the feature heatmap is used to characterize the regional distribution of dominant weather features when the model performs weather recognition. The map information of a specific weather category in the feature heatmap can be calculated, so that the region of the dominant weather feature is used as the position of the high-heat region in the feature heatmap. Therefore, the features corresponding to each position of the high-heat region have high dominant weather features.

[0048] Specifically, the feature heatmap has values related to heat, and each value is used to characterize the contribution of different regions to the final classification result. Usually, techniques such as Gard-CAM are used to emphasize the regions most influential for classification through gradient information, that is, the high-heat regions. The high-heat regions usually represent the feature regions that the model considers most important for classification decisions. Taking Gard-CAM as an example, the steps of Gard-CAM include: forward propagation, backward propagation, pooling gradients, weighted feature degrees, ReLU activation, and upsampling. Among them, in the process of forward propagation, the input image passes through a convolutional neural network to obtain an output feature map and a final classification score. In the backward propagation step, the gradient of the feature map is calculated by backpropagating the classification score of a specific category. These gradients reflect the contribution of each region in the input image to the score of this category.

[0049] Specifically, based on the feature heatmap, using the first bounding box to label the positions of corresponding high-heat regions and labeling the high-heat regions to form a first label means interpolating the feature heatmap so that each feature heatmap has the same size as the image data to be detected, marking the positions of the regions with high dominant weather features in each feature heatmap, labeling the corresponding region positions with the first bounding box, and taking the determined classification result as the first label of the first bounding box after verification. Among them, the position of the region with high dominant weather features refers to the corresponding position of the high-heat region. For example, if the high-heat region corresponding to camera 1 is (x1, y1), then the position of the first bounding box is (x1, y1). If the classification result corresponding to the first bounding box is rainy after verification, then the first label is rainy.

[0050] Specifically, generating an initial object detection dataset based on the first bounding box and the first label means forming a set of data with multiple first bounding boxes and first labels to construct an initial object detection dataset.

[0051] In an alternative embodiment, determining the position of the weather dynamic elements based on the first optical flow map includes:

[0052] Based on the first optical flow map, label the weather dynamic elements using the second target box; based on the second target box, determine the position of the weather dynamic elements, and use the weather type corresponding to the weather dynamic elements as the second label of the second target box.

[0053] Specifically, labeling the weather dynamic elements using the second target box based on the first optical flow map means using the characteristics of the optical flow map generated based on the temporal image to label the corresponding regional positions in the form of a target box. For example, label the weather characteristics corresponding to weather types such as wind and rain using the second target.

[0054] Specifically, determining the position of the weather dynamic elements based on the second target box and using the weather type corresponding to the weather dynamic elements as the second label of the second target box means using the position of the second target box in the image as the position of the corresponding weather dynamic elements, so as to use the weather type of the corresponding weather dynamic elements as the second label. For example, if the position labeled by the second target box has the dynamic elements of rain, the corresponding second label is rain.

[0055] In an alternative embodiment, based on the image data to be inspected and the second optical flow map, use the weather feature detection model to perform feature fusion to determine the weather feature target result, including:

[0056] Based on the image data to be inspected and the second optical flow map, use the weather feature detection model to perform feature fusion to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result; based on the preset confidence threshold and the confidence, screen the weather feature prediction results to determine the weather feature target result.

[0057] Specifically, screening the weather feature prediction results based on the preset confidence threshold and the confidence to determine the weather feature target result means screening the weather feature prediction results through the confidence threshold and retaining the results with a confidence above the threshold. It should be understood that the preset confidence threshold can be set according to the actual working conditions, and this embodiment does not make specific limitations on this.

[0058] In an alternative embodiment, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder.

[0059] Based on the image data to be inspected and the second optical flow map, using the weather feature detection model to perform feature fusion to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result includes:

[0060] Input the image data to be inspected at the first moment into the first image feature encoder to determine the first image feature; input the image data to be inspected at the second moment into the second image feature encoder to determine the second image feature; input the second optical flow map into the third image feature encoder to determine the third image feature; input the first image feature, the second image feature, and the third image feature into the feature fusion encoder to determine the fused feature; based on the feature decoder, decode the fused feature to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result.

[0061] Specifically, as Figure 2 shown, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. The first image feature encoder is used to extract the image feature of the image to be inspected at the first moment, that is, to extract the image feature of the pre-phase image data; the second image feature encoder is used to extract the image feature of the image to be inspected at the second moment, that is, to extract the image feature of the post-phase image data; the third image feature encoder is used to extract the image feature of the second optical flow map; the feature fusion encoder is used to fuse the extracted image features to obtain the fused image feature; the feature decoder is used to obtain the weather feature target detection frames of different sizes, that is, the weather feature target results.

[0062] Specifically, during the training of the first image feature encoder, the second image feature encoder, and the third image feature encoder using the target detection data set, the first image feature encoder and the second image feature encoder respectively learn the process of determining the initial target detection data set, that is, determining the position of the high-heat region corresponding to the first target box and the corresponding first label, and the third image feature encoder learns the process of determining the position of the weather dynamic elements, that is, determining the position corresponding to the second target box and the corresponding second label.

[0063] Specifically, during the process of inputting the image data to be inspected and the second optical flow map into the weather feature detection model for feature fusion to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result, through the first image feature encoder, the second image feature encoder, and the third image feature encoder, the weather feature detection model pays more attention to the dominant feature regions. Thus, when the image data and the optical flow map are input into the weather feature detection model as inputs, the model only needs to focus on the regions with dominant features, and since the dominant features of these regions are related to weather factors, it can ensure the accuracy of the weather feature target result while reducing the computational resource requirements.

[0064] In an alternative embodiment, determining the corresponding first optical flow map based on historical image data includes:

[0065] Based on the historical image data at the first moment and the historical image data at the second moment, using the optical flow method, determine the corresponding first optical flow map.

[0066] Specifically, based on the historical image data at the first moment and the historical image data at the second moment, using the optical flow method, determining the corresponding optical flow map means using the optical flow method to determine the corresponding optical flow map by using the phase image. For example, it is implemented by calling the calcOpticalFlow library of opencv. It should be understood that the method for determining the corresponding second optical flow map based on the image data to be inspected is the same as the method for determining the first optical flow map, and will not be elaborated here.

[0067] In this embodiment, a weather feature target inspection device is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be elaborated again. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0068] This embodiment provides a weather feature target inspection device, as Figure 3 shown, including:

[0069] A dataset determination module 201, configured to acquire continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data. The specific process can refer to the relevant description of step S102 in the above embodiment, and will not be elaborated here.

[0070] A model training module 202, configured to determine a target detection dataset based on the historical image data and the first optical flow map. The specific process can refer to the relevant description of step S103 in the above embodiment, and will not be elaborated here.

[0071] An image determination module 203, configured to train a target detection model based on the target detection dataset to obtain a trained weather feature detection model. The specific process can refer to the relevant description of step S101 in the above embodiment, and will not be elaborated here.

[0072] An image to be inspected acquisition module, configured to acquire continuously collected image data to be inspected, and determine a corresponding second optical flow map based on the image data to be inspected. The specific process can refer to the relevant description of step S104 in the above embodiment, and will not be elaborated here.

[0073] A result determination module 205, configured to perform feature fusion using the weather feature detection model based on the image data to be inspected and the second optical flow map, and determine a weather feature target result. The specific process can refer to the relevant description of step S105 in the above embodiment, and will not be elaborated here.

[0074] The weather feature target inspection device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0075] An embodiment of the present invention further provides a computer device having the above Figure 3 shown weather feature target inspection device. Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. As Figure 4 shown, the computer device includes: one or more processors 301, a memory 302, and an interface for connecting each component, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphic information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 4 One processor 301 is taken as an example in

[0076] The processor 301 can be a central processor, a network processor, or a combination thereof. Among them, the processor 301 can further include a hardware chip. The above hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a generic array logic, or any combination thereof.

[0077] Among them, the memory 302 stores instructions executable by at least one processor 301, so that the at least one processor 301 executes the method shown in the above embodiment.

[0078] The memory 302 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device and the like. In addition, the memory 302 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 302 may optionally include a memory remotely provided relative to the processor 301, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0079] The memory 302 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, a hard disk, or a solid-state drive; the memory 302 may further include a combination of the above types of memory. The computer device further includes a communication interface 303 for the computer device to communicate with other devices or communication networks.

[0080] Embodiments of the present invention also provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine-readable storage medium and to be downloaded through a network and stored in a local storage medium, so that the methods described herein can be stored in such software processed on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may further include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0081] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for testing weather feature targets, characterized in that, The method includes: Obtain historical image data collected continuously. Based on the historical image data, use the optical flow method to determine the corresponding first optical flow map; Based on the region with obvious weather features in the historical image data and the region with weather dynamic elements in the first optical flow map, determine the target detection data set; Train the target detection model based on the target detection data set to obtain a weather feature detection model that has completed training; Obtain continuously collected image data to be inspected. Based on the image data to be inspected, use the optical flow method to determine the corresponding second optical flow map; Based on the image data to be inspected and the second optical flow map, use the weather feature detection model for feature fusion to determine the weather feature target result.

2. The method according to claim 1, wherein The step of determining the target detection data set based on the region with obvious weather features in the historical image data and the region with weather dynamic elements in the first optical flow map includes: Input the historical image data into a pre-trained multi-modal model to determine the initial target detection data set; Based on the first optical flow map, determine the positions of the weather dynamic elements; Based on the initial target detection data set and the positions of the weather dynamic elements, determine the target detection data set.

3. The method according to claim 2, wherein The step of inputting the historical image data into a pre-trained multi-modal model to determine the initial target detection data set includes: Input the historical image data into a pre-trained multi-modal model to obtain a feature heat map and an image prediction classification result; Based on the feature heat map, use the first target box to label the positions of the corresponding high-heat regions, and label the high-heat regions to form the first label; Based on the first target box and the first label, generate the initial target detection data set.

4. The method according to claim 2, wherein The step of determining the positions of the weather dynamic elements based on the first optical flow map includes: Based on the first optical flow map, use the second target box to label the weather dynamic elements; Based on the second target box, determine the positions of the weather dynamic elements, and use the weather type corresponding to the weather dynamic elements as the second label of the second target box.

5. The method according to claim 1, characterized in that The step of using the weather feature detection model for feature fusion based on the image data to be inspected and the second optical flow map to determine the weather feature target result includes: Based on the image data to be inspected and the second optical flow map, use the weather feature detection model for feature fusion to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result; Based on a pre-set confidence threshold and the confidence level, filter the weather feature prediction results to determine the weather feature target result.

6. The method according to claim 5, characterized in that The weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. The step of using the weather feature detection model for feature fusion based on the image data to be inspected and the second optical flow map to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result includes: Input the image data to be inspected at the first moment into the first image feature encoder to determine the first image feature; Input the image data to be inspected at the second moment into the second image feature encoder to determine the second image features; Input the second optical flow map into the third image feature encoder to determine the third image features; Input the first image features, the second image features, and the third image features into the feature fusion encoder to determine the fused features; Based on the feature decoder, decode the fused features to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result.

7. A weather feature target inspection device, characterized in that The apparatus includes: A historical image acquisition module, configured to acquire continuously acquired historical image data, and based on the historical image data, use the optical flow method to determine the corresponding first optical flow map; A data set determination module, configured to determine a target detection data set based on the region with obvious weather features in the historical image data and the region with weather dynamic elements in the first optical flow map; A model training module, configured to train a target detection model based on the target detection data set to obtain a trained weather feature detection model; An image to be inspected acquisition module, configured to acquire continuously acquired image data to be inspected, and based on the image data to be inspected, use the optical flow method to determine the corresponding second optical flow map; A result determination module, configured to perform feature fusion using the weather feature detection model based on the image data to be inspected and the second optical flow map to determine the weather feature target result.

8. A computer device, characterized in that, Comprising: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the weather feature target inspection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the weather feature target inspection method according to any one of claims 1 to 6.

Images