Weather characteristic target inspection method, device, equipment and medium

Abstract

The invention relates to the technical field of image processing, and discloses a weather characteristic target inspection method, device, equipment and medium, and the method comprises the steps: obtaining continuously collected historical image data, and determining a corresponding first optical flow graph based on the historical image data; determining a target detection data set based on the historical image data and the first optical flow graph; training the target detection model based on the target detection data set to obtain a trained weather feature detection model; continuously acquired to-be-detected image data are acquired, and a corresponding second optical flow graph is determined based on the to-be-detected image data; and based on the to-be-detected image data and the second optical flow graph, performing feature fusion by using a weather feature detection model, and determining a weather feature target result. According to the scheme of the invention, the target detection data set determined by the image data and the optical flow graph is used as the model training data, so that the model pays attention to the region with significant weather features, and the accuracy of the weather feature target result is ensured while the computing resource demand is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a weather feature target inspection method, device, equipment and medium. Background Art

[0002] With the rapid development of computer vision technology, image classification has been widely used in many fields. Especially in weather recognition, judging weather conditions by analyzing image data has become a hot research topic. Traditional weather recognition methods mainly rely on meteorological sensors and meteorological models. Although these methods are accurate, they are costly and have high requirements on environmental conditions.

[0003] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has shown excellent performance in image classification tasks. Using CNNs for image classification has achieved remarkable results in fields such as object recognition and face recognition. However, applying this technology to weather recognition still faces some challenges, such as how to effectively deal with the impact of various lighting and weather conditions on image quality, and how to deal with the diversity of weather features in images.

[0004] In the related art, weather recognition methods mainly rely on multimodal data fusion and complex feature extraction technology, which are relatively complex to implement and have high requirements for computing resources. Therefore, there is a corresponding demand in the related art for recognition with low computing resource requirements and accurate weather forecasting. Summary of the invention

[0005] In view of this, the present invention provides a weather characteristic target inspection method, device, equipment, and medium to solve the technical problem of high computing resource requirements in related technologies.

[0006] In a first aspect, the present invention provides a weather feature target inspection method, the method comprising: obtaining continuously collected historical image data, and determining a corresponding first optical flow map based on the historical image data; determining a target detection data set based on the historical image data and the first optical flow map; training a target detection model based on the target detection data set to obtain a trained weather feature detection model; obtaining continuously collected image data to be inspected, and determining a corresponding second optical flow map based on the image data to be inspected; based on the image data to be inspected and the second optical flow map, using the weather feature detection model to perform feature fusion to determine the weather feature target result.

[0007] In combination with the first aspect, in a possible implementation of the first aspect, a target detection dataset is determined based on historical image data and a first optical flow map, including: inputting the historical image data into a pre-trained multimodal model to determine an initial target detection dataset; determining the location of weather dynamic elements based on the first optical flow map; and determining the target detection dataset based on the initial target detection dataset and the location of the weather dynamic elements.

[0008] In combination with the first aspect, in a possible implementation of the first aspect, historical image data is input into a pre-trained multimodal model to determine an initial target detection data set, including: inputting the historical image data into the pre-trained multimodal model to obtain a feature heat map and an image prediction classification result; based on the feature heat map, using a first target box to mark the position of the corresponding high thermal area, and marking the high thermal area to form a first label; based on the first target box and the first label, generating an initial target detection data set.

[0009] In combination with the first aspect, in a possible implementation method of the first aspect, the position of the weather dynamic elements is determined based on the first optical flow map, including: based on the first optical flow map, using the second target frame to mark the weather dynamic elements; based on the second target frame, the position of the weather dynamic elements is determined, and the weather type corresponding to the weather dynamic elements is used as the second label of the second target frame.

[0010] In combination with the first aspect, in a possible implementation of the first aspect, based on the image data to be inspected and the second optical flow map, a weather feature detection model is used to perform feature fusion to determine the weather feature target result, including: based on the image data to be inspected and the second optical flow map, a weather feature detection model is used to perform feature fusion to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result; based on a preset confidence threshold and confidence level, the weather feature prediction results are screened to determine the weather feature target result.

[0011] In combination with the first aspect, in a possible implementation manner of the first aspect, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. Based on the image data to be inspected and the second optical flow map, feature fusion is performed using a weather feature detection model to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result, including: inputting the image data to be inspected at a first moment into a first image feature encoder to determine the first image feature; inputting the image data to be inspected at a second moment into a second image feature encoder to determine the second image feature; inputting the second optical flow map into a third image feature encoder to determine the third image feature; inputting the first image feature, the second image feature, and the third image feature into a feature fusion encoder to determine the fusion feature; based on a feature decoder, the fusion feature is decoded to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result.

[0012] In combination with the first aspect, in a possible implementation of the first aspect, determining a corresponding first optical flow map based on historical image data includes: determining the corresponding first optical flow map using an optical flow method based on historical image data at a first moment and historical image data at a second moment.

[0013] In a second aspect, the present invention provides a weather feature target inspection device, which includes: a historical image acquisition module, used to acquire continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data; a data set determination module, used to determine a target detection data set based on the historical image data and the first optical flow map; a model training module, used to train a target detection model based on the target detection data set to obtain a trained weather feature detection model; a to-be-inspected image acquisition module, used to acquire continuously collected to-be-inspected image data, and determine a corresponding second optical flow map based on the to-be-inspected image data; and a result determination module, used to perform feature fusion using the weather feature detection model based on the to-be-inspected image data and the second optical flow map to determine the weather feature target result.

[0014] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the weather characteristic target verification method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the weather characteristic target verification method of the first aspect or any corresponding embodiment thereof.

[0016] The technical solution of the present invention has the following advantages: The present invention provides a method, device, equipment, and medium for inspecting a weather feature target. The method determines a target detection data set through historical image data and a first optical flow map, and uses the target detection data set to complete the training of a target detection model, thereby inputting the acquired image data and the optical flow map into the trained weather feature detection model, and using feature fusion to determine the weather feature target result. In this process, the target detection data set is determined through the historical image data and the first optical flow map, so that the data in the target detection data set has more explicit weather factor features than the original data, and other areas are ignored, thereby reducing the amount of data calculation, and on this basis, the target detection data set is used as the training data of the model, so that the model can master the corresponding weather factor features through training and learning, so that the weather feature detection model pays more attention to the explicit feature area, so that when the image data and the optical flow map are used as input and input into the weather feature detection model, the model only needs to pay attention to the area with explicit features, and because the explicit features of these areas are related to weather factors, the accuracy of the weather feature target results can be guaranteed while reducing the demand for computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flow chart of a weather characteristic target verification method provided according to an embodiment of the present invention; Figure 2 is a schematic diagram of a process for applying a weather feature detection model according to an embodiment of the present invention; Figure 3 is a structural block diagram of a weather characteristic target inspection device provided according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] According to an embodiment of the present invention, an embodiment of a weather characteristic target verification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0021] This embodiment provides a weather feature target verification method, such as Figure 1 As shown, the method comprises the following steps: S101. Acquire continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data.

[0022] Specifically, determining the corresponding optical flow map based on the continuously collected historical image data refers to determining the corresponding first optical flow map by using the previous phase image data and the next phase image data and the optical flow method.

[0023] Specifically, the continuously acquired historical image data refers to pre-acquired image data that is continuous in time. For example, the historical image data at the first moment and the historical image data at the second moment are two image data that are continuous in acquisition time, namely, the front phase image data and the back phase image data.

[0024] S102: Determine a target detection data set based on historical image data and the first optical flow map.

[0025] Specifically, based on the historical image data and the first optical flow map, determining the target detection data set refers to respectively determining the areas with explicit weather features in the historical image data and determining the areas with weather dynamic elements in the first optical flow map, so that the areas with explicit weather features and the areas with weather dynamic elements are collectively used as the target detection data set. It should be understood that the areas with explicit weather features and the areas with weather dynamic elements are represented in the form of target boxes, and the corresponding positions are the positions of the corresponding areas in the image, and each corresponding area has a corresponding specific weather type. Among them, the historical image data refers to the image data collected in advance, and the historical image data is used as training data.

[0026] S103: training the target detection model based on the target detection data set to obtain a trained weather feature detection model.

[0027] Specifically, training the target detection model based on the target detection data set to obtain a trained weather feature detection model means using the target detection data set as training data to train the target detection model until the target detection model converges. Generally speaking, the loss function at this time basically no longer continues to decrease or the model reaches the maximum number of iterations. It is considered that the target detection model has completed training, thereby obtaining a weather feature detection model.

[0028] S104: Acquire continuously collected image data to be inspected, and determine a corresponding second optical flow map based on the image data to be inspected.

[0029] Specifically, determining the corresponding second optical flow map based on the continuously acquired image data to be inspected refers to determining the corresponding second optical flow map by using the previous phase image data and the next phase image data and the optical flow method.

[0030] Specifically, the continuously acquired image data to be inspected refers to image data that is acquired in real time and is continuous in time. For example, the image data to be inspected at the first moment and the image data to be inspected at the second moment are two image data that are acquired continuously in time, namely, the front phase image data and the back phase image data.

[0031] S105: Based on the image data to be inspected and the second optical flow map, a weather feature detection model is used to perform feature fusion to determine a weather feature target result.

[0032] Specifically, the weather feature detection model has three image feature encoders, a feature fusion encoder and a feature decoder. The image feature encoders are used to extract image features of the previous phase image data, the subsequent phase image data and the optical flow map respectively; the feature fusion encoder is used to fuse the extracted image features to obtain fused image features; the feature decoder is used to obtain weather feature target detection frames of different sizes, that is, weather feature target results.

[0033] The present invention provides a weather feature target inspection method, device, equipment, and medium. The method determines a target detection data set through historical image data and a first optical flow map, and uses the target detection data set to complete the training of a target detection model, so that the acquired image data and the optical flow map are input into the trained weather feature detection model, and the weather feature target result is determined by feature fusion. In this process, the target detection data set is determined through the historical image data and the first optical flow map, so that the data in the target detection data set has more explicit weather factor features than the original data, and other areas are ignored, thereby reducing the amount of data calculation, and on this basis, the target detection data set is used as the training data of the model, so that the model can master the corresponding weather factor features through training and learning, so that the weather feature detection model pays more attention to the explicit feature area, so that when the image data and the optical flow map are used as input and input into the weather feature detection model, the model only needs to pay attention to the area with explicit features, and because the explicit features of these areas are related to weather factors, the accuracy of the weather feature target results can be guaranteed while reducing the demand for computing resources.

[0034] In an optional implementation, determining the target detection data set based on the historical image data and the first optical flow map includes: Input historical image data into a pre-trained multimodal model to determine an initial target detection data set; determine the location of weather dynamic elements based on the first optical flow map; and determine the target detection data set based on the initial target detection data set and the location of weather dynamic elements.

[0035] Specifically, the pre-trained multimodal model refers to a model that supports text and images, such as the CLIP (Contrastive Language-Image Pre-training) model. The pre-trained multimodal model will output an image feature layer at the output. The image feature layer determines the classification of the image through the fully connected layer, and obtains the feature heat map of the same size as the image input to the model through the branch convolution layer and the upsampling layer. The high heat area in each feature heat map and the corresponding weather type classification are the initial target detection data set. The initial target detection data set is expressed as the first target box and the weather type classification corresponding to the first target box. For example, in the historical image data collected by camera 1, the corresponding high heat area can be (x1, y1), and in the historical image data collected by camera 2, the corresponding high heat area can be (x2, y1), (x3, y2), etc. It should be understood that each camera can have one or more high heat areas. Among them, since each characteristic heat map has a corresponding classification, and the position of the high heat area in each characteristic heat map is the position with high explicit weather characteristics, the classification is related to the position of the high heat area in each characteristic heat map, thereby forming a corresponding relationship between the camera and the highly sensitive area. Taking the high heat area corresponding to camera 1 as (x1, y1) and (x2, y2) as an example, the (x1, y1) corresponding to camera 1 can be a position with high explicit weather characteristics relative to cloudy days, and the (x2, y2) corresponding to camera 1 can be a position with high explicit weather characteristics relative to sunny days.

[0036] Specifically, based on the first optical flow map, determining the position of the dynamic weather element means using the optical flow map to display the characteristics of dynamic weather changes, using the second target frame to mark the location of the area with the characteristics, and marking the weather type corresponding to the characteristics. For example: the weather type corresponding to the weather characteristic corresponding to the position marked by the second target frame is wind, and the wind will blow the camera to shake. The optical flow map will show the change characteristics of this type of wind, and the second target frame will mark the area corresponding to the change characteristics, such as (x3, y3); the weather type corresponding to the weather characteristic corresponding to the position marked by the second target frame is rain, and rain will have many oblique lines. The optical flow map will show the change characteristics of this type of rain, and the second target frame will mark the area corresponding to the change characteristics, such as (x4, y3).

[0037] Specifically, based on the position of the initial target detection dataset and the weather dynamic elements, determining the target detection dataset means taking the above-mentioned first target box, the second target box and their respective corresponding weather types as the target detection dataset, thereby taking the area with explicit weather characteristics and the area with weather dynamic elements together as the target detection dataset.

[0038] In an optional implementation, historical image data is input into a pre-trained multimodal model to determine an initial target detection data set, including: The historical image data is input into the pre-trained multimodal model to obtain the feature heat map and image prediction classification results; based on the feature heat map, the first target box is used to mark the corresponding high thermal area position, and the high thermal area is labeled to form a first label; based on the first target box and the first label, an initial target detection data set is generated.

[0039] Specifically, the feature heat map is used to characterize the regional distribution of explicit weather features when the model is performing weather identification. The map information of specific weather categories in the feature heat map can be obtained by calculation, so that the area of ​​explicit weather features is used as the high heat area position in the feature heat map. Therefore, the feature corresponding to each high heat area position has a high explicit weather feature.

[0040] Specifically, the feature heat map has heat-related values, each of which is used to characterize the contribution of different regions to the final classification result. Usually, techniques such as Gard-CAM are used to emphasize the regions that have the greatest impact on classification, that is, high heat regions, through gradient information. High heat regions usually represent the feature regions that the model considers to be the most important for classification decisions. Taking Gard-CAM as an example, the steps of Gard-CAM include: forward propagation, back propagation, pooling gradients, weighted feature degrees, ReLU activation, and upsampling. Among them, in the process of forward propagation, the input image is passed through a convolutional neural network to obtain the output feature map and the final classification score. In the back propagation step, the gradient of the feature map is calculated by back propagating the classification score of a specific category. These gradients reflect the contribution of each region in the input image to the score of that category.

[0041] Specifically, based on the characteristic heat map, the first target frame is used to mark the corresponding high heat area position, and the high heat area is marked. Forming the first label means interpolating the characteristic heat map so that each characteristic heat map has a size consistent with the image data to be inspected, marking the area position with high dominant weather features in each characteristic heat map, marking the corresponding area position with the first target frame, and after review, the determined classification result is used as the first label of the first target frame. Among them, the area position of the highly dominant weather feature refers to the corresponding position of the high heat area. For example, the high heat area corresponding to camera 1 is (x1, y1), then the position of the first target frame is (x1, y1), if the classification result corresponding to the first target frame is rainy after review, then the first label is rainy.

[0042] Specifically, based on the first target frame and the first label, generating the initial target detection data set refers to forming a set of data having multiple first target frames and first labels to construct the initial target detection data set.

[0043] In an optional implementation, determining the position of the weather dynamic element based on the first optical flow map includes: Based on the first optical flow map, the second target frame is used to mark the weather dynamic elements; based on the second target frame, the position of the weather dynamic elements is determined, and the weather type corresponding to the weather dynamic elements is used as the second label of the second target frame.

[0044] Specifically, based on the first optical flow map, using the second target frame to mark the dynamic weather elements means using the characteristics of the optical flow map generated based on the phase image to mark the corresponding area position in the form of a target frame. For example, the weather characteristics corresponding to weather types such as wind and rain are marked using the second target.

[0045] Specifically, based on the second target frame, determining the position of the weather dynamic element, and using the weather type corresponding to the weather dynamic element as the second label of the second target frame means using the position of the second target frame in the image as the position of the corresponding weather dynamic element, and thus using the weather type corresponding to the weather dynamic element as the second label. For example, if the position marked by the second target frame has the dynamic element of rain, the corresponding second label is rain.

[0046] In an optional implementation, based on the image data to be inspected and the second optical flow map, a weather feature detection model is used to perform feature fusion to determine a weather feature target result, including: Based on the image data to be inspected and the second optical flow map, the weather feature detection model is used to perform feature fusion to determine the weather feature prediction results and the confidence level corresponding to the weather feature prediction results; based on the preset confidence threshold and confidence level, the weather feature prediction results are screened to determine the weather feature target results.

[0047] Specifically, based on the preset confidence threshold and confidence, the weather feature prediction results are screened, and determining the weather feature target result means screening the weather feature prediction results by the confidence threshold, and retaining the results with confidence above the threshold. It should be understood that the preset confidence threshold can be set according to actual working conditions, and this embodiment does not specifically limit this.

[0048] In an optional implementation, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. Based on the image data to be inspected and the second optical flow map, the weather feature detection model is used to perform feature fusion to determine the weather feature prediction result and the confidence level corresponding to the weather feature prediction result, including: The image data to be inspected at the first moment is input into the first image feature encoder to determine the first image feature; the image data to be inspected at the second moment is input into the second image feature encoder to determine the second image feature; the second optical flow map is input into the third image feature encoder to determine the third image feature; the first image feature, the second image feature, and the third image feature are input into the feature fusion encoder to determine the fusion feature; based on the feature decoder, the fusion feature is decoded to determine the weather feature prediction result and the confidence corresponding to the weather feature prediction result.

[0049] Specifically, Figure 2 As shown, the weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. The first image feature encoder is used to extract the image features of the image to be detected at the first moment, that is, to extract the image features of the image data of the previous phase; the second image feature encoder is used to extract the image features of the image to be detected at the second moment, that is, to extract the image features of the image data of the later phase; the third image feature encoder is used to extract the image features of the second optical flow map; the feature fusion encoder is used to fuse the extracted image features to obtain fused image features; the feature decoder is used to obtain weather feature target detection frames of different sizes, that is, weather feature target results.

[0050] Specifically, in the process of training the first image feature encoder, the second image feature encoder, and the third image feature encoder using the target detection dataset, the first image feature encoder and the second image feature encoder respectively learn the process of determining the initial target detection dataset, that is, determining the position of the high thermal area corresponding to the first target frame and the corresponding first label, and the third image feature encoder learns the process of determining the position of the weather dynamic elements, that is, determining the position corresponding to the second target frame and the corresponding second label.

[0051] Specifically, in the process of inputting the image data to be inspected and the second optical flow map into the weather feature detection model for feature fusion, and determining the weather feature prediction results and the confidence corresponding to the weather feature prediction results, the weather feature detection model is made to pay more attention to the explicit feature areas through the first image feature encoder, the second image feature encoder, and the third image feature encoder. Therefore, when the image data and the optical flow map are used as input and input into the weather feature detection model, the model only needs to pay attention to the areas with explicit features, and because the explicit features of these areas are related to weather factors, the accuracy of the weather feature target results can be guaranteed while reducing the demand for computing resources.

[0052] In an optional implementation, determining the corresponding first optical flow map based on the historical image data includes: Based on the historical image data at the first moment and the historical image data at the second moment, a corresponding first optical flow map is determined using an optical flow method.

[0053] Specifically, based on the historical image data of the first moment and the historical image data of the second moment, using the optical flow method, determining the corresponding optical flow map means determining the corresponding optical flow map using the phase image by the optical flow method. For example, it is implemented by calling the opencv library calcOpticalFlow. It should be understood that the method of determining the corresponding second optical flow map based on the image data to be tested is the same as the method of determining the first optical flow map, and will not be repeated here.

[0054] In this embodiment, a weather characteristic target verification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0055] This embodiment provides a weather characteristic target inspection device, such as Figure 3 As shown, including: The data set determination module 201 is used to obtain the continuously collected historical image data and determine the corresponding first optical flow map based on the historical image data. The specific process can be found in the description of step S102 in the above embodiment, which will not be repeated here.

[0056] The model training module 202 is used to determine the target detection data set based on the historical image data and the first optical flow map. The specific process can be found in the description of step S103 in the above embodiment, which will not be repeated here.

[0057] The image determination module 203 is used to train the target detection model based on the target detection data set to obtain a trained weather feature detection model. The specific process can be found in the description of step S101 in the above embodiment, which will not be repeated here.

[0058] The image acquisition module is used to acquire the continuously collected image data to be inspected, and determine the corresponding second optical flow map based on the image data to be inspected. The specific process can be found in the description of step S104 in the above embodiment, which will not be repeated here.

[0059] The result determination module 205 is used to perform feature fusion based on the image data to be detected and the second optical flow map using the weather feature detection model to determine the weather feature target result. The specific process can be found in the description of step S105 in the above embodiment, which will not be repeated here.

[0060] The weather characteristic target inspection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0061] The embodiment of the present invention also provides a computer device having the above Figure 3 The weather characteristics target inspection device shown. Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 301, memory 302, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Each component utilizes different buses to communicate with each other, and can be installed on a common mainboard or installed in other ways as required. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 301 is taken as an example.

[0062] The processor 301 may be a central processing unit, a network processor or a combination thereof. The processor 301 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0063] The memory 302 stores instructions executable by at least one processor 301, so that the at least one processor 301 executes the method shown in the above embodiment.

[0064] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 302 may optionally include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0065] The memory 302 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 302 may also include a combination of the above types of memory. The computer device also includes a communication interface 303, which is used for the computer device to communicate with other devices or communication networks.

[0066] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0067] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A weather characteristic target inspection method, characterized in that: The method comprises: Acquire continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data; Determine a target detection data set based on the historical image data and the first optical flow map; The target detection model is trained based on the target detection data set to obtain a trained weather feature detection model; Acquire continuously collected image data to be inspected, and determine a corresponding second optical flow map based on the image data to be inspected; Based on the image data to be inspected and the second optical flow map, the weather feature detection model is used to perform feature fusion to determine the weather feature target result.

2. The method according to claim 1, characterized in that The determining of the target detection data set based on the historical image data and the first optical flow map includes: Inputting the historical image data into a pre-trained multimodal model to determine an initial target detection data set; Based on the first optical flow map, determining the position of the weather dynamic element; Based on the initial target detection data set and the position of the weather dynamic element, a target detection data set is determined.

3. The method according to claim 2, characterized in that The step of inputting the historical image data into a pre-trained multimodal model to determine an initial target detection data set includes: Inputting the historical image data into a pre-trained multimodal model to obtain a feature heat map and image prediction classification results; Based on the characteristic heat map, a first target frame is used to mark the position of the corresponding high heat area, and the high heat area is marked to form a first label; Based on the first target box and the first label, an initial target detection data set is generated.

4. The method according to claim 2, characterized in that: The determining the position of the weather dynamic element based on the first optical flow map includes: Based on the first optical flow map, using a second target frame to mark weather dynamic elements; Based on the second target frame, the position of the weather dynamic element is determined, and the weather type corresponding to the weather dynamic element is used as the second label of the second target frame.

5. The method according to claim 1, characterized in that The step of performing feature fusion based on the image data to be inspected and the second optical flow map using the weather feature detection model to determine the weather feature target result includes: Based on the image data to be inspected and the second optical flow map, the weather feature detection model is used to perform feature fusion to determine a weather feature prediction result and a confidence level corresponding to the weather feature prediction result; Based on a preset confidence threshold and the confidence level, the weather characteristic prediction results are screened to determine a weather characteristic target result.

6. The method according to claim 5, characterized in that The weather feature detection model includes: a first image feature encoder, a second image feature encoder, a third image feature encoder, a feature fusion encoder, and a feature decoder. Based on the image data to be inspected and the second optical flow map, the weather feature detection model is used to perform feature fusion, and a weather feature prediction result and a confidence level corresponding to the weather feature prediction result are determined, including: Inputting the image data to be inspected at a first moment into the first image feature encoder to determine a first image feature; Inputting the image data to be inspected at a second moment into the second image feature encoder to determine a second image feature; Inputting the second optical flow map into the third image feature encoder to determine a third image feature; Inputting the first image feature, the second image feature, and the third image feature into the feature fusion encoder to determine a fusion feature; Based on the feature decoder, the fused feature is decoded to determine a weather feature prediction result and a confidence level corresponding to the weather feature prediction result.

7. The method according to claim 1, characterized in that The determining the corresponding first optical flow map based on the historical image data comprises: Based on the historical image data at the first moment and the historical image data at the second moment, a corresponding first optical flow map is determined using an optical flow method.

8. A weather characteristic target inspection device, characterized in that: The device comprises: A historical image acquisition module, used to acquire continuously collected historical image data, and determine a corresponding first optical flow map based on the historical image data; A data set determination module, configured to determine a target detection data set based on the historical image data and the first optical flow map; A model training module, used to train the target detection model based on the target detection data set to obtain a trained weather feature detection model; The image acquisition module is used to acquire the continuously collected image data to be inspected, and determine the corresponding second optical flow map based on the image data to be inspected; A result determination module is used to perform feature fusion based on the image data to be inspected and the second optical flow map using the weather feature detection model to determine the weather feature target result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the weather characteristic target verification method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the weather characteristic target verification method according to any one of claims 1 to 7.

Images