Method, device and storage medium for controlling video image device on power transmission line

Abstract

This invention provides a control method, device, and storage medium for a video imaging device on a power transmission line. The method includes: recording a first illuminance of the external environment, the brightness value of the image data, and the first power generation of a photovoltaic solar panel when a camera collects image data of the power transmission line during a first historical time period; querying a second illuminance of the external environment for a future second time period; generating a first temporal feature representing photoelectric conversion based on the first illuminance and the first power generation; generating a second temporal feature representing light shading and uncertainty based on the first illuminance and the brightness value; calculating a second power generation of the photovoltaic solar panel in the second time period based on the first temporal feature, the second temporal feature, the uncertainty, and the second illuminance; and adjusting the operating mode of the video imaging device in the second time period based on the second power generation and the remaining battery power. This embodiment effectively improves the utilization rate of battery power.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision, and in particular relates to a control method, device and storage medium for a video image device on a power transmission line. Background Technology

[0002] With the continuous expansion of the power grid and the ongoing technological upgrades, the demand for intelligent online inspection in the substation, transmission, and distribution sectors is showing a continuous growth trend. Currently, video imaging devices are being installed on transmission lines to conduct remote video / image inspections of key locations such as transmission line corridors, line equipment (such as insulators, vibration dampers, conductors, etc.), and tower foundations.

[0003] The video imaging device is equipped with a battery. Based on the battery's real-time SOC (State of Charge) level, the video imaging monitoring device is controlled to enter different operating modes, such as high-performance mode, normal mode, and low-power mode.

[0004] However, this control mode mainly relies on the relationship between the battery's real-time SOC and the boundary values ​​of the charge range, resulting in a relatively simple control dimension and low utilization of battery power. Summary of the Invention

[0005] In view of this, the present invention provides a control method, device and storage medium for a video imaging device on a power transmission line, so as to improve the utilization rate of battery power in the video imaging device on the power transmission line.

[0006] A first aspect of the present invention provides a control method for a video imaging device on a power transmission line, the video imaging device having a camera and a battery, and connected to a photovoltaic solar panel, the method comprising:

[0007] During the first historical time period, when the camera collects image data of the power transmission line, it records the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel.

[0008] Query the second illuminance of the external environment within the second future time period;

[0009] A power generation prediction model is determined; the power generation prediction model includes a first branch structure, a second branch structure, and a third branch structure;

[0010] In the first branch structure, the first time-series characteristic of photoelectric conversion is represented by the first illuminance and the first power generation.

[0011] In the second branch structure, a second temporal feature and uncertainty representing light occlusion are generated based on the first illuminance and the brightness value;

[0012] In the third branch structure, the second power generation of the photovoltaic solar panel in the second time period is calculated based on the first time series characteristics, the second time series characteristics, the uncertainty, and the second illuminance.

[0013] The operating mode of the video imaging device during the second time period is adjusted based on the second power generation and the remaining power of the battery.

[0014] A second aspect of the present invention provides a control device for a video imaging device on a power transmission line, the video imaging device having a camera and a battery, and connected to a photovoltaic solar panel, the device comprising:

[0015] The historical data recording module is used to record the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel when the camera collects image data of the power transmission line in the first historical time period.

[0016] The future data query module is used to query the second illuminance of the external environment in the second time period in the future;

[0017] A power generation prediction model determination module is used to determine a power generation prediction model; the power generation prediction model includes a first branch structure, a second branch structure, and a third branch structure;

[0018] The first feature extraction module is used to generate a first temporal feature representing photoelectric conversion based on the first illuminance and the first power generation in the first branch structure.

[0019] The second feature extraction module is used to generate a second temporal feature and uncertainty representing light occlusion based on the first illuminance and the brightness value in the second branch structure.

[0020] A power generation calculation module is used to calculate the second power generation of the photovoltaic solar panel in the second time period based on the first time-series characteristics, the second time-series characteristics, the uncertainty and the second illuminance in the third branch structure.

[0021] The operation mode adjustment module is used to adjust the operation mode of the video imaging device during the second time period based on the second power generation and the remaining power of the battery.

[0022] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a control method for a video image device on a transmission line as described in the first aspect above.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method for a video image device on a transmission line as described in the first aspect above.

[0024] A fifth aspect of the present invention provides a computer program product that, when run on a computer, causes the computer to perform the control method for a video image device on a transmission line as described in the first aspect above.

[0025] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0026] In this embodiment, when the camera collects image data of the transmission line during the first historical time period, it records the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel; it queries the second illuminance of the external environment in the second future time period; it determines the power generation prediction model; the power generation prediction model includes a first branch structure, a second branch structure, and a third branch structure; in the first branch structure, a first temporal feature representing photoelectric conversion is generated based on the first illuminance and the first power generation; in the second branch structure, a second temporal feature representing light shading and uncertainty is generated based on the first illuminance and the brightness value; in the third branch structure, the second power generation of the photovoltaic solar panel in the second time period is calculated based on the first temporal feature, the second temporal feature, the uncertainty, and the second illuminance; and the operating mode of the video image device in the second time period is adjusted based on the second power generation and the remaining battery power. This embodiment focuses on local illumination and occlusion of the video imaging device to achieve personalized power generation prediction, effectively improving the accuracy of power generation prediction. It proactively integrates the future power generation of the battery with the real-time remaining power to select a suitable operating mode for the video imaging device, enriching the control dimensions and making the operating mode closer to the future state of the video imaging device. This effectively improves the utilization rate of battery power and ensures the safe operation of transmission lines. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a control method for a video image device on a power transmission line provided in an embodiment of the present invention;

[0029] Figure 2This is a schematic diagram of the structure of a video imaging device on a power transmission line provided in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the structure of a power generation prediction model provided in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of a control device for a video image device on a power transmission line according to an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0033] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of the present application.

[0034] The technical solution of the present invention will be illustrated below through specific embodiments.

[0035] Reference Figure 1 The diagram illustrates a control method for a video image device on a power transmission line according to an embodiment of the present invention, which may specifically include the following steps:

[0036] Step 101: When the camera collects image data of the power transmission line during the first historical time period, record the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel.

[0037] In this embodiment, multiple video imaging devices can be installed along the power transmission line.

[0038] For example, such as Figure 2 As shown, the pan-tilt unit in the video imaging device is a single-arm pan-tilt unit, which supports 360° horizontal rotation and ±90° vertical rotation. A fixed channel camera is installed at the base position. The pan-tilt camera cabin is equipped with a pan-tilt wide-angle camera, a pan-tilt zoom camera, and a rear-view camera, and supports wipers and fill lights.

[0039] The pan-tilt wide-angle camera has a resolution of ≥2 million pixels and a field of view of 100°, providing a wide monitoring perspective for large-scale monitoring of the environment surrounding power transmission lines.

[0040] The pan-tilt zoom camera has a resolution of ≥4 million pixels and a 40X optical zoom capability, enabling clear close-up shots of distant power transmission line equipment for easy observation of equipment details.

[0041] The rear-view camera has a resolution of ≥2 million pixels to supplement the monitoring perspective and ensure that the device can provide comprehensive coverage of all key areas of the power transmission line.

[0042] Video imaging devices contain batteries, such as lithium batteries.

[0043] The video imaging device is connected to a photovoltaic solar panel, which charges the battery.

[0044] In this embodiment, during the period when the historical camera collects image data of the power transmission line (i.e., within the first time period), on the one hand, the weather information of the external environment of the video image device is queried and recorded from the cloud. The weather information includes the first illuminance at each time point in the first time period. On the other hand, the brightness value of each frame of image data is counted (represented by the average brightness value of each pixel). The brightness values ​​of multiple frames of image data form a time series. Furthermore, the first power generation of the photovoltaic solar panel at each time point in the first time period is queried.

[0045] The initial illuminance of the external environment of the video imaging device, the brightness value of the image data, and the initial power generation of the photovoltaic solar panel can all be normalized using methods such as Max-Min (maximum-min).

[0046] Generally, weather information is released for a relatively wide area. However, objects such as clouds, trees, and towers may block the sunlight received by photovoltaic solar panels. The amount of sunlight received by photovoltaic solar panels varies depending on their location. Since the distance between video imaging devices (especially cameras) and photovoltaic solar panels is relatively close, the amount of sunlight perceived by the video imaging devices (especially cameras) is usually the same as or similar to the amount of sunlight received by the photovoltaic solar panels. Therefore, the brightness value of the image data can reflect the amount of sunlight received by the photovoltaic solar panels to a certain extent, supplementing the initial illuminance in the weather information.

[0047] Step 102: Query the second illuminance of the external environment in the second future time period.

[0048] In this embodiment, the future weather information of the external environment of the video imaging device can be queried and recorded from the cloud. This weather information includes the second illuminance in the second time period in the future.

[0049] For the secondary illuminance of the external environment of the video image device, normalization can be performed using methods such as Max-Min (maximum-min).

[0050] Step 103: Determine the power generation prediction model.

[0051] In this embodiment, a power generation prediction model can be loaded, wherein, as... Figure 3 As shown, the power generation prediction model includes the first branch structure, the second branch structure, and the third branch structure.

[0052] Step 104: In the first branch structure, a first time-series characteristic representing photoelectric conversion is generated based on the first illuminance and the first power generation.

[0053] In this embodiment, the first illuminance and the first power generation can be input into the first branch structure, and a first timing feature representing photoelectric conversion can be generated based on the first illuminance and the first power generation.

[0054] In a design, such as Figure 3 As shown, the first branch structure includes a first fully connected layer and a first convolutional layer. Therefore, in this design, a shape adjustment operation Reshape can be performed on the first power generation to obtain a two-dimensional first power generation feature map.

[0055] The first illuminance is mapped into the first fully connected layer to the second illuminance. The second illuminance is then reshaped to obtain a two-dimensional first illuminance feature map, capturing complex attenuation features.

[0056] In the first convolutional layer, the first power generation feature map is convolved with the first illumination feature map as the kernel to obtain the second power generation feature map.

[0057] Perform a shape adjustment operation Reshape on the second power generation feature map to obtain a one-dimensional representation of the photoelectric conversion timing feature.

[0058] In this design, the first illumination feature map, which is reconstructed from the ambient illumination sequence through full connection, is used as the dynamic convolution kernel. Convolution operation is performed on the second power generation feature map after the power generation sequence is reconstructed. This can adaptively complete illumination normalization and power feature matching, and directly, accurately, and robustly extract photoelectric conversion features, thereby achieving efficient characterization of the aging, degradation, shading, and fault status of solar photovoltaic panels.

[0059] In addition, the first branch has a lightweight structure, simple operators, and omits the learning convolution kernel, making it suitable for deployment on the edge.

[0060] Step 105: In the second branch structure, generate a second temporal feature and uncertainty representing illumination occlusion based on the first illuminance and brightness value.

[0061] In this embodiment, the first illuminance and luminance values ​​can be input into the first branch structure, and a second temporal feature and uncertainty representing light occlusion can be generated based on the first illuminance and luminance values.

[0062] Among them, light shading may be fixed objects such as trees and dust / ice on solar photovoltaic panels, or moving objects such as clouds and water droplets on solar photovoltaic panels. The second time series feature can measure the overall light shading, and uncertainty can measure the fluctuation of the shading object between fixed and moving.

[0063] In a design, such as Figure 3 As shown, the second branch structure includes a second fully connected layer, a third fully connected layer, and a second convolutional layer. Therefore, in this design, a shape adjustment operation Reshape can be performed on the brightness value to obtain a two-dimensional first brightness feature map.

[0064] The first illuminance is input into the second fully connected layer and mapped to the third illuminance. The third illuminance is then subjected to a shape adjustment operation (Reshape) to obtain a two-dimensional second illuminance feature map.

[0065] In the second convolutional layer, the first brightness feature map is convolved with the second illumination feature map as the kernel to obtain the second brightness feature map.

[0066] Perform a shape adjustment operation Reshape on the second brightness feature map to obtain a one-dimensional representation of the second temporal feature of illumination occlusion.

[0067] The second brightness feature is input into the third fully connected layer and mapped to represent the uncertainty of illumination occlusion.

[0068] In this design, by mapping the ambient lighting sequence to a dynamic adaptive convolution kernel through a fully connected layer and performing convolution operations with image brightness features, illumination normalization and adaptive extraction of occlusion features are achieved. This not only outputs explicit illumination occlusion features but also quantifies the uncertainty of occlusion through a fully connected layer, realizing integrated modeling of occlusion feature recognition and uncertainty assessment. It combines the advantages of physical priors and data-driven approaches to improve the accuracy and reliability of occlusion detection, making it particularly suitable for intelligent operation and maintenance in complex outdoor lighting environments.

[0069] In addition, the second branch has a lightweight structure, simple operators, and omits the learning convolution kernel, making it suitable for deployment on the edge.

[0070] In practical applications, since the output (i.e. uncertainty) of a neural network is data-driven and may not necessarily conform to the physical laws of photovoltaics, real physical measurements can be used to correct the output (i.e. uncertainty) of the neural network, so that physical priors and data-driven processes are deeply integrated, thereby improving the accuracy of the output (i.e. uncertainty) of the neural network.

[0071] In practical implementation, for the same point in time, the first average value of the first illuminance and the second average value of the luminance can be statistically analyzed. The ratio between the first index value and the second index value is calculated to obtain the third index value. Among them, the first index value is the difference between the first average value and the second average value, and the second index value is the sum of the first average value and the second average value. That is, the third index value = (the first average value of the first illuminance - the second average value of the luminance) / (the first average value of the first illuminance + the second average value of the luminance). The third index value reflects the imbalance between macroscopic ambient light and microscopic fixed-point light.

[0072] The uncertainty is corrected by using the average value of the third indicator and other methods such as weighted summation, so that the uncertainty of the neural network output can truly reflect the discriminability of the occlusion state.

[0073] Step 106: In the third branch structure, calculate the second power generation of the photovoltaic solar panel in the second time period based on the first time series characteristics, the second time series characteristics, the uncertainty and the second illuminance.

[0074] In this embodiment, the first time-series feature, the second time-series feature, the uncertainty and the second illuminance input third branch structure are combined to predict the second power generation of the photovoltaic solar panel in the second time period by integrating information such as photoelectric conversion, illuminance shading and future weather.

[0075] In one embodiment of the present invention, such as Figure 3 As shown, the third branch structure includes a first encoder, a second encoder, an attention module, and a fourth fully connected layer; therefore, in this embodiment, step 106 may include the following steps:

[0076] Step 1061: Concatenate the first temporal feature with the second illuminance to form the first original multimodal feature.

[0077] In this embodiment, the Concat function can be used to concatenate the first temporal feature with the second illuminance to form a new feature, denoted as the first original multimodal feature.

[0078] Step 1062: Input the first original multimodal features into the first encoder to extract the first target multimodal features.

[0079] In this embodiment, the first encoder includes structures such as convolutional layers and LSTM (Long Short-Term Memory) networks, which can input the first original multimodal features into the first encoder to extract temporal features and obtain the first target multimodal features.

[0080] Step 1063: Concatenate the second temporal feature, uncertainty and second illuminance to form the second original multimodal feature.

[0081] In this embodiment, the Concat function can be used to concatenate the second temporal feature, uncertainty, and second illuminance into a second original multimodal feature.

[0082] Step 1064: Input the second original multimodal features into the second encoder to extract the second target multimodal features.

[0083] In this embodiment, the second encoder includes structures such as convolutional layers and LSTM (Long Short-Term Memory) networks, which can input the second original multimodal features into the second encoder to extract temporal features and obtain the second target multimodal features.

[0084] Step 1065: Input the first target multimodal features and the second target multimodal features into the attention module and fuse them into the third target multimodal features.

[0085] In this embodiment, the first target multimodal feature and the second target multimodal feature are input into the attention module, and the attention mechanism is used to fuse the first target multimodal feature and the second target multimodal feature into a new feature, which is denoted as the third target multimodal feature.

[0086] In a design, such as Figure 3 As shown, the attention module includes a first attention layer, a second attention layer, a third attention layer, and a gating network.

[0087] In this design, the first target multimodal features are input into the first attention layer, and the first attention multimodal features are generated using a self-attention mechanism. The self-attention mechanism uses the first target multimodal features themselves as the Q (query) matrix, K (key) matrix, and V (value) matrix. It can focus on the most recent photoelectric conversion efficiency and the continuous change trend of future illumination, explore the lag effect of photoelectric conversion, and realize the information purification of the first target multimodal features.

[0088] The second target multimodal features are input into the second attention layer, and the second attention multimodal features are generated using a self-attention mechanism. The self-attention mechanism uses the second target multimodal features themselves as the Q (query) matrix, K (key) matrix, and V (value) matrix. It can focus on the temporal propagation law of illumination occlusion and the sudden change moment of illumination occlusion uncertainty, explore the superposition effect of occlusion and illumination, and realize the information purification of the second target multimodal features.

[0089] The importance of each piece of information in the first and second attentional multimodal features is dynamically weighted and then fused into a third attentional multimodal feature by inputting the first and second attentional multimodal features into the gating network.

[0090] The third attention multimodal features are input into the third attention layer, and the third target multimodal features are generated using a self-attention mechanism. The self-attention mechanism uses the third attention multimodal features themselves as the Q (query) matrix, K (key) matrix, and V (value) matrix, which can capture long-term dependencies across features, extract global key decision information, and improve the accuracy of prediction.

[0091] Step 1066: Input the third target multimodal features into the fourth fully connected layer and map them as the second power generation of the photovoltaic solar panel in the second time period.

[0092] In this embodiment, the third target multimodal features can be input into the fourth fully connected layer to perform the prediction task, and the third target multimodal features can be mapped to the second power generation of the photovoltaic solar panel at each time point in the second time period.

[0093] Step 107: Adjust the operating mode of the video imaging device in the second time period based on the second power generation and the remaining battery power.

[0094] In this embodiment, the SOC of the video imaging device battery can be queried, and the operating mode of the video imaging device in the second time period can be adjusted based on the second power generation and the battery SOC.

[0095] In one embodiment of the present invention, step 107 may include the following steps:

[0096] Step 1071: Integrate the second power generation during the second time period to obtain the estimated charging amount.

[0097] In this embodiment, the second power generation can be integrated over time within the second time period to obtain the estimated charge amount of the battery.

[0098] Step 1072: Set the safe power level based on the first time-series characteristics, the second time-series characteristics, and the uncertainty.

[0099] In this embodiment, a safe battery level can be set based on the first timing characteristics, the second timing characteristics, and the changing patterns of uncertainty, to ensure sufficient power to cope with situations such as the issuance of temporary tasks or sudden safety accidents.

[0100] In its implementation, the power generation prediction model also includes a first-head structure and a second-head structure. Both the first-head structure and the second-head structure include fully connected layers, activation functions, and other structures. The first-head structure is used to classify the first level of photoelectric conversion, and the second-head structure is used to classify the second level of light shading.

[0101] The first branch structure and the first head structure can be trained jointly. That is, the first branch structure and the first head structure can form a complete multi-classification model. The output of the first branch structure (the first temporal feature) is the input of the first head structure. Cross-entropy can be used as the loss function and Adam (Adaptive momentum) as the optimizer to independently conduct supervised training of the multi-classification model. When the training is completed, the parameters of the first branch structure and the parameters of the first head structure remain unchanged.

[0102] The second branch structure and the second head structure are jointly trained. That is, the second branch structure and the second head structure can form a complete multi-branch model. One branch is a classification structure and the other branch is a regression structure. The second head structure is a classification structure and the third fully connected layer is a regression structure. The output of the second branch structure (the second time-series feature) is the input of the second head structure. The cross-entropy and mean squared error can be used together (e.g., weighted summation) as loss functions and Adam as the optimizer to independently conduct supervised training on the multi-branch model. When the training is completed, the parameters of the second branch structure and the parameters of the second head structure remain unchanged.

[0103] When training the third branch structure, the first, second, and third branch structures together form the power generation prediction model. The mean absolute error can be used as the loss function and Adam as the optimizer to independently conduct supervised training of the power generation prediction model. During the training process, the parameters of the first and second branch structures remain unchanged, while the parameters of the third branch structure are updated.

[0104] On the one hand, the first temporal features are input into the first head structure for classification to obtain the first level representing photoelectric conversion.

[0105] On the other hand, the second temporal features are input into the second head structure for classification to obtain the second level representing illumination occlusion.

[0106] The power generation scenario is determined by using methods such as table lookup (piecewise function) based on the average values ​​of multiple first-level values, multiple second-level values, and multiple uncertain values. Each power generation scenario is configured with a safe power capacity, thereby determining the safe power capacity of the current battery according to the current power generation scenario.

[0107] For example, the first level includes high conversion, medium conversion, and low conversion; the second level includes slight shading, moderate shading, and severe shading; and the uncertainty includes low fluctuation, medium fluctuation, and high fluctuation. If the first level is high conversion, the second level is slight shading, and the uncertainty is low fluctuation, it indicates high reliability (good sunlight, slight shading, and stable prediction). If the power generation scenario belongs to a high power generation potential scenario, the safe power capacity can be set to 15%. If the first level is medium conversion, the second level is moderate shading, and the uncertainty is medium fluctuation, it indicates high reliability (average sunlight, moderate shading, and stable prediction). If the power generation scenario belongs to a medium power generation potential scenario, the safe power capacity can be set to 25%, and so on.

[0108] Step 1073: Subtract the safety charge from the total charge to obtain the margin charge.

[0109] In this embodiment, the total power can be determined. The total power is the sum of the estimated charging amount and the remaining power of the battery. The safety power is subtracted from the total power to obtain the margin power, which is the power that can be allocated to various tasks in the second time period.

[0110] Step 1074: Determine the operating mode of the video imaging device in the second time period according to the margin of power.

[0111] In this embodiment, the operating mode of the video imaging device in the second time period can be determined according to the power margin. The power consumption of the operating mode is positively correlated with the power margin. That is, the more power margin there is, the higher the power consumption of the operating mode can be selected to release the performance of the video imaging device. Conversely, the lower the power margin, the lower the power consumption of the operating mode can be selected to prioritize the critical services provided by the video imaging device.

[0112] For example, the operating modes of the video imaging device include deep sleep, low-power sensing, normal operation, and high performance. Devices activated in deep sleep include RTC (real-time clock) and low-power MCU (microcontroller unit), with power consumption <10μA. Devices activated in low-power sensing include PIR (passive infrared sensor) and ambient light sensor, supporting wake-up interrupt, with power consumption of 50μA-1mA. Devices activated in normal operation include down-clocked (e.g., 8MHz) CPU (central processing unit), supporting LoRa (remote) communication, with each sensor capable of periodic sampling, and power consumption of 1mA-50mA. Devices activated in high performance include a full-featured CPU, high-definition camera, and AI (artificial intelligence) inference, supporting mobile cellular communication, with power consumption of 50mA-500mA.

[0113] In this embodiment, when the camera collects image data of the transmission line during the first historical time period, it records the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel; it queries the second illuminance of the external environment in the second future time period; it determines the power generation prediction model; the power generation prediction model includes a first branch structure, a second branch structure, and a third branch structure; in the first branch structure, a first temporal feature representing photoelectric conversion is generated based on the first illuminance and the first power generation; in the second branch structure, a second temporal feature representing light shading and uncertainty is generated based on the first illuminance and the brightness value; in the third branch structure, the second power generation of the photovoltaic solar panel in the second time period is calculated based on the first temporal feature, the second temporal feature, the uncertainty, and the second illuminance; and the operating mode of the video image device in the second time period is adjusted based on the second power generation and the remaining battery power. This embodiment focuses on local illumination and occlusion of the video imaging device to achieve personalized power generation prediction, effectively improving the accuracy of power generation prediction. It proactively integrates the future power generation of the battery with the real-time remaining power to select a suitable operating mode for the video imaging device, enriching the control dimensions and making the operating mode closer to the future state of the video imaging device. This effectively improves the utilization rate of battery power and ensures the safe operation of transmission lines.

[0114] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0115] Reference Figure 4 The diagram illustrates a control device for a video imaging device on a power transmission line according to an embodiment of the present invention. The video imaging device includes a camera and a battery, and is connected to a photovoltaic solar panel. The device may specifically include the following modules:

[0116] The historical data recording module 401 is used to record the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel when the camera collects image data of the power transmission line during the first historical time period.

[0117] Future data query module 402 is used to query the second illuminance of the external environment in the second time period in the future;

[0118] The power generation prediction model determination module 403 is used to determine the power generation prediction model; the power generation prediction model includes a first branch structure, a second branch structure and a third branch structure;

[0119] The first feature extraction module 404 is used to generate a first temporal feature representing photoelectric conversion based on the first illuminance and the first power generation in the first branch structure.

[0120] The second feature extraction module 405 is used to generate a second temporal feature and uncertainty representing light occlusion in the second branch structure based on the first illuminance and the brightness value.

[0121] The power generation calculation module 406 is used to calculate the second power generation of the photovoltaic solar panel in the second time period based on the first time series characteristics, the second time series characteristics, the uncertainty and the second illuminance in the third branch structure.

[0122] The operation mode adjustment module 407 is used to adjust the operation mode of the video image device during the second time period based on the second power generation and the remaining power of the battery.

[0123] In one embodiment of the present invention, the first branch structure includes a first fully connected layer and a first convolutional layer, and the first feature extraction module 404 is further configured to:

[0124] A shape adjustment operation is performed on the first power generation to obtain a two-dimensional first power generation feature map;

[0125] The first illuminance is input into the first fully connected layer and mapped to the second illuminance.

[0126] The second illuminance is subjected to a shape adjustment operation to obtain a two-dimensional first illuminance feature map;

[0127] In the first convolutional layer, the first power generation feature map is convolved with the first illumination feature map as the convolution kernel to obtain the second power generation feature map.

[0128] A shape adjustment operation is performed on the second power generation feature map to obtain a one-dimensional representation of the photoelectric conversion timing feature.

[0129] In one embodiment of the present invention, the second branch structure includes a second fully connected layer, a third fully connected layer, and a second convolutional layer, and the second feature extraction module 405 is further configured to:

[0130] A shape adjustment operation is performed on the brightness value to obtain a two-dimensional first brightness feature map;

[0131] The first illuminance is input into the second fully connected layer and mapped to the third illuminance.

[0132] The third illuminance is subjected to a shape adjustment operation to obtain a two-dimensional second illuminance feature map;

[0133] In the second convolutional layer, the first brightness feature map is convolved with the second illumination feature map as the convolution kernel to obtain the second brightness feature map.

[0134] Perform a shape adjustment operation on the second brightness feature map to obtain a one-dimensional second temporal feature representing illumination occlusion;

[0135] The second brightness feature is input into the third fully connected layer and mapped to represent the uncertainty of light occlusion.

[0136] In one embodiment of the present invention, the second feature extraction module 405 is further configured to:

[0137] Calculate the first average value of the first illuminance and the second average value of the brightness value;

[0138] Calculate the ratio between the first indicator value and the second indicator value to obtain the third indicator value; the first indicator value is the difference between the first average value and the second average value, and the second indicator value is the sum of the first average value and the second average value;

[0139] The uncertainty is corrected using the average value of the third indicator.

[0140] In one embodiment of the present invention, the third branch structure includes a first encoder, a second encoder, an attention module, and a fourth fully connected layer; the power generation calculation module 406 is further configured to:

[0141] The first temporal feature and the second illuminance are concatenated to form the first original multimodal feature;

[0142] The first original multimodal features are input into the first encoder to extract the first target multimodal features;

[0143] The second temporal feature, the uncertainty, and the second illuminance are concatenated to form the second original multimodal feature;

[0144] The second original multimodal features are input into the second encoder to extract the second target multimodal features;

[0145] The first target multimodal feature and the second target multimodal feature are input into the attention module and fused into a third target multimodal feature;

[0146] The third target multimodal feature is input into the fourth fully connected layer and mapped to the second power generation of the photovoltaic solar panel in the second time period.

[0147] In one embodiment of the present invention, the attention module includes a first attention layer, a second attention layer, a third attention layer, and a gating network; the power generation calculation module 406 is further configured to:

[0148] The first target multimodal features are input into the first attention layer, and the first attention multimodal features are generated using a self-attention mechanism;

[0149] The second target multimodal features are input into the second attention layer, and a second attention multimodal features are generated using a self-attention mechanism;

[0150] The first attentional multimodal feature and the second attentional multimodal feature are input into the gating network and fused into a third attentional multimodal feature;

[0151] The third attentional multimodal features are input into the third attention layer, and a self-attention mechanism is used to generate the third target multimodal features.

[0152] In one embodiment of the present invention, the operating mode adjustment module 407 is further configured to:

[0153] Integrate the second power generation during the second time period to obtain the estimated charging amount;

[0154] A safe power level is set based on the first timing feature, the second timing feature, and the uncertainty.

[0155] Subtract the safety charge from the total charge to obtain the margin charge; the total charge is the sum of the estimated charge and the remaining charge of the battery.

[0156] The operating mode of the video imaging device in the second time period is determined according to the margin of power consumption; the power consumption of the operating mode is positively correlated with the margin of power consumption.

[0157] In one embodiment of the present invention, the power generation prediction model further includes a first head structure and a second head structure, wherein the first branch structure and the first head structure are jointly trained, and the second branch structure and the second head structure are jointly trained; the operation mode adjustment module 407 is further configured to:

[0158] The first temporal feature is input into the first head structure for classification to obtain the first level of photoelectric conversion.

[0159] The second temporal feature is input into the second head structure for classification to obtain the second level of light occlusion.

[0160] The power generation scenario is determined by the average of multiple first-level values, the average of multiple second-level values, and the average of multiple uncertainties.

[0161] Determine the safe power volume according to the described power generation scenario.

[0162] The present invention provides a control device for a video image device on a power transmission line. By using the control device for the video image device on a power transmission line, the various steps in the aforementioned control method embodiments for the video image device on a power transmission line can be implemented.

[0163] It should be noted that the module division in the control devices of various video image devices on transmission lines provided in the above embodiments is illustrative and only represents one logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0164] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the embodiments of the present invention can be embodied in the form of a computer program product, which is stored in a computer storage medium and includes several instructions to cause an electronic device or processor to execute all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned computer storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] Furthermore, the control device for the video image device on the transmission line provided in the above embodiments and the control method embodiment for the video image device on the transmission line belong to the same concept. For details of its specific implementation process, please refer to the method embodiment, which will not be repeated here.

[0166] Reference Figure 5 The diagram illustrates an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the control method embodiment of the video image device on the transmission line described above. Alternatively, when the processor executes the computer program, it implements the functions of each module in the control device embodiment of the video image device on the transmission line described above.

[0167] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which can be used to describe the execution process of the computer program in the electronic device.

[0168] The electronic device may be a desktop computer, a cloud server, or other computing device. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 This is merely one example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0169] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0170] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0171] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the control method for a video image device on a transmission line as described in the foregoing embodiments.

[0172] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method for a video image device on a transmission line as described in the foregoing embodiments.

[0173] This invention also discloses a computer program product that, when run on a computer, causes the computer to execute the control method for the video image device on the transmission line described in the foregoing embodiments.

[0174] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of controlling a video image device on a power transmission line, characterized by, The video imaging device includes a camera and a battery, and is connected to a photovoltaic solar panel. The method includes: During the first historical time period, when the camera collects image data of the power transmission line, it records the first illuminance of the external environment, the brightness value of the image data, and the first power generation of the photovoltaic solar panel. Query the second illuminance of the external environment within the second future time period; A power generation prediction model is determined; the power generation prediction model includes a first branch structure, a second branch structure, and a third branch structure; In the first branch structure, the first time-series characteristic of photoelectric conversion is represented by the first illuminance and the first power generation. In the second branch structure, a second temporal feature and uncertainty representing light occlusion are generated based on the first illuminance and the brightness value; In the third branch structure, the second power generation of the photovoltaic solar panel in the second time period is calculated based on the first time series characteristics, the second time series characteristics, the uncertainty, and the second illuminance. The operating mode of the video imaging device during the second time period is adjusted based on the second power generation and the remaining power of the battery. The third branch structure includes a first encoder, a second encoder, an attention module, and a fourth fully connected layer; the calculation of the second power generation of the photovoltaic solar panel in the second time period based on the first time-series characteristics, the second time-series characteristics, the uncertainty, and the second illuminance in the third branch structure includes: The first temporal feature and the second illuminance are concatenated to form the first original multimodal feature; The first original multimodal features are input into the first encoder to extract the first target multimodal features; The second temporal feature, the uncertainty, and the second illuminance are concatenated to form the second original multimodal feature; The second original multimodal features are input into the second encoder to extract the second target multimodal features; The first target multimodal feature and the second target multimodal feature are input into the attention module and fused into a third target multimodal feature; The third target multimodal feature is input into the fourth fully connected layer and mapped to the second power generation of the photovoltaic solar panel in the second time period.

2. The method of claim 1, wherein the method further comprises: The first branch structure includes a first fully connected layer and a first convolutional layer. In the first branch structure, generating a first temporal characteristic representing photoelectric conversion based on the first illuminance and the first power generation includes: A shape adjustment operation is performed on the first power generation to obtain a two-dimensional first power generation feature map; The first illuminance is input into the first fully connected layer and mapped to the second illuminance. The second illuminance is subjected to a shape adjustment operation to obtain a two-dimensional first illuminance feature map; In the first convolutional layer, the first power generation feature map is convolved with the first illumination feature map as the convolution kernel to obtain the second power generation feature map. A shape adjustment operation is performed on the second power generation feature map to obtain a one-dimensional representation of the photoelectric conversion timing feature.

3. The method of claim 1, wherein the method further comprises: The second branch structure includes a second fully connected layer, a third fully connected layer, and a second convolutional layer. In the second branch structure, a second temporal feature and uncertainty representing illumination occlusion are generated based on the first illuminance and the brightness value, including: A shape adjustment operation is performed on the brightness value to obtain a two-dimensional first brightness feature map; The first illuminance is input into the second fully connected layer and mapped to the third illuminance. The third illuminance is subjected to a shape adjustment operation to obtain a two-dimensional second illuminance feature map; In the second convolutional layer, the first brightness feature map is convolved with the second illumination feature map as the convolution kernel to obtain the second brightness feature map. Perform a shape adjustment operation on the second brightness feature map to obtain a one-dimensional second temporal feature representing illumination occlusion; The second brightness feature is input into the third fully connected layer and mapped to represent the uncertainty of light occlusion.

4. The method of claim 3, wherein the method further comprises: In the second branch structure, generating a second temporal feature and uncertainty representing illumination occlusion based on the first illuminance and the brightness value further includes: Calculate the first average value of the first illuminance and the second average value of the brightness value; Calculate the ratio between the first indicator value and the second indicator value to obtain the third indicator value; the first indicator value is the difference between the first average value and the second average value, and the second indicator value is the sum of the first average value and the second average value; The uncertainty is corrected using the average value of the third indicator.

5. The method of claim 1, wherein the method further comprises: The attention module includes a first attention layer, a second attention layer, a third attention layer, and a gating network; The step of fusing the first target multimodal features and the second target multimodal features into the attention module to form a third target multimodal feature includes: The first target multimodal features are input into the first attention layer, and the first attention multimodal features are generated using a self-attention mechanism; The second target multimodal features are input into the second attention layer, and a second attention multimodal features are generated using a self-attention mechanism; The first attentional multimodal feature and the second attentional multimodal feature are input into the gating network and fused into a third attentional multimodal feature; The third attentional multimodal features are input into the third attention layer, and a self-attention mechanism is used to generate the third target multimodal features.

6. The method of claim 1-5, wherein, Adjusting the operating mode of the video imaging device during the second time period based on the second power generation and the remaining battery power includes: Integrate the second power generation during the second time period to obtain the estimated charging amount; A safe power level is set based on the first timing feature, the second timing feature, and the uncertainty. Subtract the safety charge from the total charge to obtain the margin charge; the total charge is the sum of the estimated charge and the remaining charge of the battery. The operating mode of the video imaging device in the second time period is determined according to the margin of power consumption; the power consumption of the operating mode is positively correlated with the margin of power consumption.

7. The method of claim 6, wherein the method further comprises: The power generation prediction model further includes a first head structure and a second head structure, the first branch structure and the first head structure are jointly trained, and the second branch structure and the second head structure are jointly trained; the step of setting a safe power quantity based on the first time series characteristics, the second time series characteristics and the uncertainty includes: The first temporal feature is input into the first head structure for classification to obtain the first level of photoelectric conversion. The second temporal feature is input into the second head structure for classification to obtain the second level of light occlusion. The power generation scenario is determined by the average of multiple first-level values, the average of multiple second-level values, and the average of multiple uncertainties. Determine the safe power volume according to the described power generation scenario.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the control method for a video image device on a power transmission line as described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by the processor, it implements the control method for a video image device on a power transmission line as described in any one of claims 1-7.

Images