A multi-modal ice and snow recognition method and system for power transmission monitoring

Abstract

The present invention belongs to the technical field of electric power equipment monitoring, and specifically provides a multimodal ice and snow recognition method and system for power transmission monitoring, comprising: obtaining real-time image data by using a power transmission monitoring camera, and obtaining environmental data at the power transmission monitoring camera through an environmental sensor; preprocessing the obtained real-time image data and environmental data; inputting the preprocessed real-time image data and environmental data into a pre-stored multimodal ice and snow recognition model to predict whether a lens is covered with ice and snow and the degree of ice and snow coverage; controlling the lens to perform ice and snow melting processing according to the predicted ice and snow coverage of the lens; the present invention can predict the ice and snow coverage of the lens in real time, and automatically perform ice melting processing, thereby avoiding the degradation of shooting quality and safety hazards caused by untimely human intervention or operational errors.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power equipment monitoring, and in particular relates to a multi-modal ice and snow recognition method and system for power transmission monitoring. Background Art

[0002] As winter deepens, the frequent appearance of ice and snow weather has brought unprecedented challenges to the operation of power transmission monitoring equipment. Under extreme ice and snow conditions, the lens of the power transmission monitoring camera is often covered with thick ice and snow, which not only greatly affects the shooting quality and reduces the image clarity, but also may cause the power inspection personnel to be unable to accurately judge the line status due to the obstruction of vision, thus affecting the safe and stable operation of the power system.

[0003] Traditionally, the problem of ice and snow covering the lens of power transmission monitoring equipment has mainly relied on manual intervention to remove ice and snow. This method usually heats the lens of the monitoring equipment to melt the ice and snow based on the weather forecast in the area. However, this manual intervention method has many shortcomings. On the one hand, manual intervention cannot accurately grasp the timing of heating, and heating is often performed only when the ice and snow have already caused serious impact on shooting. This not only leads to excessive loss of battery power, but also may miss the best time to remove ice and snow due to untimely heating. On the other hand, due to human factors, such as operational errors or neglect, heating, wipers and other ice and snow removal actions may not be performed in time, further exacerbating the impact of ice and snow on power transmission monitoring equipment.

[0004] In addition, because the power transmission monitoring system belongs to the local area network, it cannot directly access the real-time weather forecast data from the external network for security reasons. This means that even if the weather forecast from the external network has a high degree of accuracy, it cannot be directly used to guide the snow and ice removal work of the power transmission monitoring equipment. Therefore, how to achieve real-time monitoring and intelligent processing of the snow and ice coverage of the power transmission monitoring equipment lens without relying on external network weather data has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a multimodal ice and snow recognition method and system for power transmission monitoring to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a multimodal ice and snow recognition method for power transmission monitoring, comprising:

[0007] Use the power transmission monitoring camera to obtain real-time image data, and use the environmental sensor to obtain the environmental data at the power transmission monitoring camera;

[0008] Preprocess the acquired real-time image data and environmental data;

[0009] The pre-processed real-time image data and environmental data are input into the pre-stored multimodal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage;

[0010] According to the predicted lens coverage of ice and snow, the lens is controlled to melt ice and snow.

[0011] A further improvement of the technical solution is that the image data includes image data acquired by the power transmission monitoring camera under different lighting conditions, image data acquired by the power transmission monitoring camera under different angles, and image data acquired by the power transmission monitoring camera under different ice and snow thickness conditions;

[0012] Environmental data include meteorological data, atmospheric electric field data and location data. Meteorological data include atmospheric pressure, atmospheric temperature, atmospheric humidity, wind temperature, real-time wind direction, real-time wind speed and precipitation. Atmospheric electric field data include electric field strength and field strength changes. Location data includes the longitude and latitude of the power transmission monitoring camera.

[0013] A further improvement of the technical solution is that the data set for training the multimodal ice and snow recognition model uses multiple modal data, including image data and environmental data collected by various sensors, specifically:

[0014] The multimodal training dataset is denoted as ;in, It is i data samples, including two modal data, where Indicates i image data, is the image data sample space; It is i Environmental data, including g The environmental data vector collected by the sensors is denoted as ,in Indicates j Environmental data vector collected by sensors; is the sample space of environmental data; It is i The class labels of the data samples; , is the class label for no ice or snow, is the class label for snow-covered ice; m is the number of samples.

[0015] A further improvement of the technical solution is that the training method of the pre-stored multimodal ice and snow recognition model includes:

[0016] Leveraging multimodal training datasets The multimodal ice and snow recognition model is jointly trained, and two strategies are adopted to fuse the multimodal data, namely feature-level fusion and decision-level fusion;

[0017] Based on two different strategies, in the multimodal training dataset The above are trained separately to obtain two multimodal ice and snow recognition models, which are respectively denoted as feature-level fusion ice and snow recognition model and decision-level fusion ice and snow recognition model.

[0018] Further improvements of the technical solution include preprocessing and feature extraction of image data and environmental data, and generating a feature-level fusion ice and snow recognition model based on feature-level fusion training, and the method specifically includes:

[0019] The formula for feature extraction of image datasets based on ResNet50 neural network or ViT neural network is:

[0020] ;

[0021] in, For image data Image feature vector extracted by ResNet50 or ViT;

[0022] Standardize and preprocess the environmental data set to obtain the environmental feature vector ;

[0023] The transformer-based multimodal feature cross-fusion method is used to fuse the environment feature vector and the image feature vector to obtain the fused feature vector, whose calculation formula is:

[0024] ;

[0025] in, is the environmental feature vector, For image data Image feature vector extracted by ResNet50 or ViT, is the fusion feature vector of the environment feature vector and the image feature vector; It is a transformer-based multimodal feature cross-fusion method;

[0026] Based on the fusion feature vector training, a feature-level fusion ice and snow recognition model is generated, and its expression is:

[0027] ;

[0028] in, Yes The prediction results, It is a multimodal ice and snow recognition model with feature-level fusion. is the fusion feature vector of the environment feature vector and the image feature vector, are the model parameters of the multimodal snow and ice recognition model with feature-level fusion;

[0029] The minimization objective function of the feature-level fusion snow and ice recognition model is:

[0030] ;

[0031] in, is the minimization objective function of the feature-level snow and ice recognition model, It is i The class labels of the data samples, It is a multimodal ice and snow recognition model based on feature-level fusion right The prediction results, m is the number of samples.

[0032] A further improvement of the technical solution is to use environmental data and image data to train the corresponding classification models respectively, and obtain a decision-level fusion ice and snow recognition model based on decision-level fusion, and the method specifically includes:

[0033] Using the gradient boosting tree classification algorithm, the environmental data is trained to obtain the gradient boosting tree classification model , whose expression is:

[0034] ;

[0035] Use the ResNet50 algorithm or ViT algorithm to train the image data to obtain the image prediction model , whose expression is:

[0036] or

[0037] ;

[0038] right and Perform decision-level weighted fusion to obtain a decision-level fusion ice and snow recognition model, which is expressed as follows:

[0039] ;

[0040] in, represents the decision-level fusion snow and ice recognition model, is the model parameter of the decision-level fusion snow and ice recognition model, It is a gradient boosting tree classification model generated by environmental data training. It is an image prediction model generated by training image data. is the prediction result of the decision-level fusion snow and ice recognition model. and is the weight coefficient;

[0041] The minimization objective function of the decision-level fusion snow and ice recognition model is:

[0042] ;

[0043] in, is the minimization objective function of the decision-level fusion snow and ice recognition model, is the cross entropy target loss of the image prediction model, is the cross entropy target loss of the gradient boosted tree classification model, is the weight factor.

[0044] In a second aspect, the present invention provides a multi-modal ice and snow recognition system for power transmission monitoring, comprising:

[0045] A data acquisition module is used to acquire real-time image data using a power transmission monitoring camera, and to acquire environmental data at the power transmission monitoring camera through an environmental sensor;

[0046] A data preprocessing module is used to preprocess the acquired real-time image data and environmental data;

[0047] A multimodal ice and snow recognition module is used to input the pre-processed real-time image data and environmental data into a pre-stored multimodal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage;

[0048] The ice and snow melting module is used to control the lens to melt ice and snow according to the predicted ice and snow coverage of the lens.

[0049] In a third aspect, a terminal is provided, including:

[0050] processor, memory, wherein:

[0051] The memory is used to store computer programs.

[0052] The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.

[0053] According to a fourth aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the methods described in the above aspects.

[0054] The beneficial effect of the present invention is that the present invention can predict the ice and snow coverage of the lens in real time and automatically perform ice melting processing, thereby avoiding the degradation of shooting quality and safety hazards caused by untimely human intervention or operational errors. This solution does not rely on real-time weather forecast data from the external network, but only uses micro-weather data and real-time image data of the local environment of the lens to perform ice and snow identification and ice melting processing, so it has strong adaptability and reliability.

[0055] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0058] Figure 2 is a schematic block diagram of a system according to an embodiment of the present invention.

[0059] Figure 3 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention.

[0060] 210 is a data acquisition module, 220 is a data preprocessing module, 230 is a multimodal ice and snow recognition module, and 240 is an ice and snow melting module. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0063] The multimodal ice and snow recognition method for power transmission monitoring provided in the embodiment of the present invention is executed by a computer device, and accordingly, the multimodal ice and snow recognition system for power transmission monitoring runs in the computer device.

[0064] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a multi-modal ice and snow identification system for power transmission monitoring. According to different requirements, the order of the steps in the flowchart may be changed, and some may be omitted.

[0065] like Figure 1 As shown, the method includes:

[0066] Step 110, using a power transmission monitoring camera to obtain real-time image data, and obtaining environmental data at the power transmission monitoring camera through an environmental sensor;

[0067] Step 120, preprocessing the acquired real-time image data and environmental data;

[0068] Step 130, inputting the pre-processed real-time image data and environmental data into a pre-stored multi-modal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage;

[0069] Step 140, controlling the lens to perform ice and snow melting processing according to the predicted ice and snow coverage of the lens.

[0070] To facilitate understanding of the present invention, the multimodal ice and snow recognition method for power transmission monitoring provided by the present invention is further described below based on the principle of the multimodal ice and snow recognition method for power transmission monitoring provided by the present invention, combined with the multimodal ice and snow recognition process for power transmission monitoring in the embodiment.

[0071] Specifically, the image data includes image data acquired by the power transmission monitoring camera under different lighting conditions, image data acquired by the power transmission monitoring camera at different angles, and image data acquired by the power transmission monitoring camera under different ice and snow thickness conditions; the environmental data includes meteorological data, atmospheric electric field data and location data; the meteorological data includes atmospheric pressure, atmospheric temperature, atmospheric humidity, wind temperature, real-time wind direction, real-time wind speed and precipitation; the atmospheric electric field data includes electric field strength and field strength changes; the location data includes the longitude and latitude of the power transmission monitoring camera.

[0072] Among them, the data set for training the multimodal ice and snow recognition model uses multiple modal data, including image data and environmental data collected by various sensors, specifically:

[0073] The multimodal training dataset is denoted as ;in, It is idata samples, including two modal data, where Indicates i image data, is the image data sample space; It is i Environmental data, including g The environmental data vector collected by the sensors is denoted as ,in Indicates j Environmental data vector collected by sensors; is the sample space of environmental data; It is i The class labels of the data samples; , , is the class label for no ice or snow, is the class label for snow-covered ice; m is the number of samples.

[0074] In addition, based on the above multimodal training dataset ,The training method of the pre-existing multimodal ice and snow recognition model includes:

[0075] Leveraging multimodal training datasets The multimodal ice and snow recognition model is jointly trained, and two strategies are adopted to fuse the multimodal data, namely feature-level fusion and decision-level fusion;

[0076] Based on two different strategies, in the multimodal training dataset The above are trained separately to obtain two multimodal ice and snow recognition models, which are respectively denoted as feature-level fusion ice and snow recognition model and decision-level fusion ice and snow recognition model.

[0077] Specifically, the image data and environmental data are preprocessed and features are extracted, and a feature-level fusion ice and snow recognition model is generated based on feature-level fusion training. The method specifically includes:

[0078] The formula for feature extraction of image datasets based on ResNet50 neural network or ViT neural network is:

[0079] ;

[0080] in, For image data Image feature vector extracted by ResNet50 or ViT;

[0081] Standardize and preprocess the environmental data set to obtain the environmental feature vector ;

[0082] The transformer-based multimodal feature cross-fusion method is used to fuse the environment feature vector and the image feature vector to obtain the fused feature vector, whose calculation formula is:

[0083] ;

[0084] in, is the environmental feature vector, For image data Image feature vector extracted by ResNet50 or ViT, is the fusion feature vector of the environment feature vector and the image feature vector; It is a transformer-based multimodal feature cross-fusion method;

[0085] Based on the fusion feature vector training, a feature-level fusion ice and snow recognition model is generated, and its expression is:

[0086] ;

[0087] in, Yes The prediction results, It is a multimodal ice and snow recognition model with feature-level fusion. is the fusion feature vector of the environment feature vector and the image feature vector, are the model parameters of the multimodal snow and ice recognition model with feature-level fusion;

[0088] The minimization objective function of the feature-level fusion snow and ice recognition model is:

[0089] ;

[0090] in, is the minimization objective function of the feature-level snow and ice recognition model, It is i The class labels of data samples, that is, The true class label, include , It is a multimodal ice and snow recognition model based on feature-level fusion right The prediction results, m is the number of samples.

[0091] In addition, the corresponding classification models are trained using environmental data and image data respectively, and a decision-level fusion ice and snow recognition model is obtained based on decision-level fusion. The method specifically includes:

[0092] Using the gradient boosting tree classification algorithm, the environmental data is trained to obtain the gradient boosting tree classification model , whose expression is:

[0093] ;

[0094] Use the ResNet50 algorithm or ViT algorithm to train the image data to obtain the image prediction model , whose expression is:

[0095] or

[0096] ;

[0097] right and Perform decision-level weighted fusion to obtain a decision-level fusion ice and snow recognition model, which is expressed as follows:

[0098] ;

[0099] in, represents the decision-level fusion snow and ice recognition model, is the model parameter of the decision-level fusion snow and ice recognition model, It is a gradient boosting tree classification model generated by environmental data training. It is an image prediction model generated by training image data. is the prediction result of the decision-level fusion snow and ice recognition model. and is the weight coefficient;

[0100] The minimization objective function of the decision-level fusion snow and ice recognition model is:

[0101] ;

[0102] in, is the minimization objective function of the decision-level fusion snow and ice recognition model, is the cross entropy target loss of the image prediction model, is the cross entropy target loss of the gradient boosted tree classification model, is the weight factor.

[0103] Input the collected real-time image data and environmental data into the trained multimodal ice and snow recognition model or , respectively according to or Make real-time predictions and output prediction results The snow and ice melting module is based on the multimodal snow and ice recognition model. or The prediction results , control the wiper and heating operations, and perform automatic ice melting.

[0104] In some embodiments, the multimodal ice and snow identification system 200 for power transmission monitoring may include multiple functional modules composed of computer program segments. The computer program of each program segment in the multimodal ice and snow identification system 200 for power transmission monitoring may be stored in the memory of a computer device and executed by at least one processor to execute (see Figure 1 Description) Multi-modal ice and snow recognition function for power transmission monitoring.

[0105] In this embodiment, the multi-modal ice and snow identification system 200 for power transmission monitoring can be divided into multiple functional modules according to the functions it performs, such as Figure 2 As shown. The functional modules may include: a data acquisition module 210, a data preprocessing module 220, a multimodal ice and snow recognition module 230 and an ice and snow melting module 240. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0106] Among them, the data acquisition module is used to obtain real-time image data using the power transmission monitoring camera, and to obtain environmental data at the power transmission monitoring camera through the environmental sensor; the data preprocessing module is used to preprocess the acquired real-time image data and environmental data; the multimodal ice and snow recognition module is used to input the preprocessed real-time image data and environmental data into a pre-stored multimodal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage; the ice and snow ablation module is used to control the lens to perform ice and snow ablation processing according to the predicted ice and snow coverage of the lens.

[0107] Figure 3 A schematic diagram of the structure of a terminal 300 provided in an embodiment of the present invention, wherein the terminal 300 can be used to execute the multi-modal ice and snow recognition method for power transmission monitoring provided in an embodiment of the present invention.

[0108] The terminal 300 may include: a processor 310, a memory 320 and a communication module 330. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0109] The memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can perform some or all of the steps in the following method embodiments.

[0110] The processor 310 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 310 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0111] The communication module 330 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.

[0112] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0113] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0114] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0115] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0116] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0118] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A multi-modal ice and snow recognition method for power transmission monitoring, characterized in that: include: Use the power transmission monitoring camera to obtain real-time image data, and use the environmental sensor to obtain the environmental data at the power transmission monitoring camera; Preprocess the acquired real-time image data and environmental data; Input the pre-processed real-time image data and environmental data into the pre-stored multimodal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage; Control the lens to melt the ice and snow according to the predicted lens coverage of ice and snow; The dataset for training the multimodal ice and snow recognition model uses multiple modal data, including image data and environmental data collected by various sensors, specifically: The multimodal training dataset is denoted as ;in, It is i data samples, including two modal data, where Indicates i image data, is the image data sample space; It is i Environmental data, including g The environmental data vector collected by the sensors is denoted as ,in Indicates j Environmental data vector collected by sensors; is the sample space of environmental data; It is i The class labels of the data samples; , is the class label for no ice or snow, is the class label for snow-covered ice; m is the number of samples; The training methods of the pre-existing multimodal ice and snow recognition model include: Leveraging multimodal training datasets The multimodal ice and snow recognition model is jointly trained, and two strategies are adopted to fuse the multimodal data, namely feature-level fusion and decision-level fusion; Based on two different strategies, in the multimodal training dataset The above two models are trained separately to obtain two multimodal ice and snow recognition models, which are respectively recorded as feature-level fusion ice and snow recognition model and decision-level fusion ice and snow recognition model; A feature-level fusion ice and snow recognition model is generated based on feature-level fusion training, and the method includes: Extract features from image datasets based on ResNet50 neural network or ViT neural network; Standardize and preprocess the environmental data set to obtain the environmental feature vector ; The transformer-based multimodal feature cross-fusion method is used to fuse the environment feature vector and the image feature vector to obtain a fused feature vector. Generate a feature-level fusion snow and ice recognition model based on fusion feature vector training; A decision-level fusion ice and snow recognition model is obtained based on decision-level fusion, and the method includes: Using the gradient boosting tree classification algorithm, the environmental data is trained to obtain a gradient boosting tree classification model; Use the ResNet50 algorithm or the ViT algorithm to train the image data to obtain an image prediction model; The gradient boosting tree classification model and the image prediction model are weighted fused at the decision level to obtain a decision level fusion ice and snow recognition model.

2. The multi-modal ice and snow identification method for power transmission monitoring and shooting according to claim 1 is characterized in that: The image data includes image data acquired by the power transmission monitoring camera under different lighting conditions, image data acquired by the power transmission monitoring camera at different angles, and image data acquired by the power transmission monitoring camera under different ice and snow thickness conditions; Environmental data include meteorological data, atmospheric electric field data and location data. Meteorological data include atmospheric pressure, atmospheric temperature, atmospheric humidity, wind temperature, real-time wind direction, real-time wind speed and precipitation. Atmospheric electric field data include electric field strength and field strength changes. Location data includes the longitude and latitude of the power transmission monitoring camera.

3. The multi-modal ice and snow identification method for power transmission monitoring and shooting according to claim 1 is characterized in that: The formula for feature extraction of image datasets based on ResNet50 neural network or ViT neural network is: ； in, For image data Image feature vector extracted by ResNet50 or ViT; The transformer-based multimodal feature cross-fusion method is used to fuse the environment feature vector and the image feature vector to obtain the fused feature vector, whose calculation formula is: ； in, is the environmental feature vector, For image data Image feature vector extracted by ResNet50 or ViT, is the fusion feature vector of the environment feature vector and the image feature vector; It is a transformer-based multimodal feature cross-fusion method; Based on the fusion feature vector training, a feature-level fusion ice and snow recognition model is generated, and its expression is: ； in, Yes The prediction results, It is a multimodal ice and snow recognition model based on feature-level fusion. is the fusion feature vector of the environment feature vector and the image feature vector, are the model parameters of the multimodal snow and ice recognition model with feature-level fusion; The minimization objective function of the feature-level fusion snow and ice recognition model is: ； in, is the minimization objective function of the feature-level snow and ice recognition model, It is i The class labels of the data samples, It is a multimodal ice and snow recognition model based on feature-level fusion right The prediction results, m is the number of samples.

4. The multi-modal ice and snow identification method for power transmission monitoring according to claim 1 is characterized in that: Using the gradient boosting tree classification algorithm, the environmental data is trained to obtain the gradient boosting tree classification model , whose expression is: ； Use the ResNet50 algorithm or ViT algorithm to train the image data to obtain the image prediction model , whose expression is: or ； right and Perform decision-level weighted fusion to obtain a decision-level fusion ice and snow recognition model, which is expressed as follows: ； in, represents the decision-level fusion snow and ice recognition model, is the model parameter of the decision-level fusion snow and ice recognition model, It is a gradient boosting tree classification model generated by environmental data training. It is an image prediction model generated by training image data. is the prediction result of the decision-level fusion snow and ice recognition model. and is the weight coefficient; The minimization objective function of the decision-level fusion snow and ice recognition model is: ； in, is the minimization objective function of the decision-level fusion snow and ice recognition model, is the cross entropy target loss of the image prediction model, is the cross entropy target loss of the gradient boosted tree classification model, is the weight factor.

5. A system based on the multimodal ice and snow recognition method according to any one of claims 1 to 4, characterized in that: include: A data acquisition module is used to acquire real-time image data using a power transmission monitoring camera, and to acquire environmental data at the power transmission monitoring camera through an environmental sensor; A data preprocessing module is used to preprocess the acquired real-time image data and environmental data; A multimodal ice and snow recognition module is used to input the pre-processed real-time image data and environmental data into a pre-stored multimodal ice and snow recognition model to predict whether the lens is covered with ice and snow and the degree of ice and snow coverage; The ice and snow melting module is used to control the lens to melt ice and snow according to the predicted ice and snow coverage of the lens.

6. A terminal, characterized in that: include: processor; A memory for storing execution instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images