Photovoltaic power station operation and maintenance method, system and medium based on multimodal data fusion

Abstract

The present invention discloses a photovoltaic power station operation and maintenance method, system and medium based on multimodal data fusion, the photovoltaic power station includes photovoltaic components, convergence components, inverter components, and detection components, the method includes: obtaining multimodal data corresponding to the photovoltaic power station collected based on the detection component in the current operation and maintenance cycle, the multimodal data includes text data, image data, audio data and video data after format conversion; and then according to the fusion processing of the multimodal data, the photovoltaic components, convergence components and inverter components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle. The present invention makes the fusion processing of each data more convenient by fusing at least two items of the multimodal data and converting the multimodal data into a unified data format, thereby realizing fast, comprehensive and accurate operation and maintenance of the photovoltaic components, convergence components and inverter components in the photovoltaic power station.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic power generation, and in particular to a photovoltaic power station operation and maintenance method, system and medium based on multimodal data fusion. Background Art

[0002] With the reduction of traditional fuel energy and the harm to the environment, renewable clean energy has received more and more attention. Photovoltaic power generation is an important development direction of renewable clean energy. It is a technology that uses the photovoltaic effect of semiconductor interfaces to directly convert light energy into electrical energy.

[0003] Generally, photovoltaic power stations are mainly composed of photovoltaic modules, bus modules and inverter modules. Abnormal operation of these components will have a great impact on the production efficiency of the power station, such as reduced power generation efficiency, increased safety risks, shortened equipment life, equipment damage, data abnormalities, etc. Therefore, it is particularly important to conduct regular inspections and maintenance of photovoltaic modules, bus modules and inverter modules to promptly detect operating faults and ensure the normal operation of equipment.

[0004] At present, the operation and maintenance of photovoltaic power stations mostly rely on on-site personnel supplemented by detection equipment. However, the detection methods of detection equipment are usually relatively simple, which makes it difficult to fully reflect the actual operation of photovoltaic components, bus components and inverter components in photovoltaic power stations, resulting in incomplete and inaccurate operation and maintenance. At the same time, there are also problems such as different communication protocols used by different detection equipment and differences in data formats, which makes data processing cumbersome. Therefore, how to quickly, comprehensively and accurately operate and maintain photovoltaic components, bus components and inverter components in photovoltaic power stations is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the present invention is to provide a photovoltaic power station operation and maintenance method, system and medium based on multimodal data fusion, aiming to solve the technical problem of how to quickly, comprehensively and accurately operate and maintain photovoltaic components, bus components and inverter components in photovoltaic power stations.

[0006] To achieve the above object, the present invention provides a photovoltaic power station operation and maintenance method based on multimodal data fusion, the photovoltaic power station includes a photovoltaic component, a bus component, an inverter component, and a detection component, and the photovoltaic power station operation and maintenance method includes:

[0007] Acquire multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, wherein the multimodal data includes text data, image data, audio data, and video data that have been converted into a format; wherein:

[0008] The text data includes: photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic component, confluence data of the confluence component, inversion data, heat dissipation data and oil level data of the inversion component;

[0009] The image data includes: a photovoltaic image of the photovoltaic component, a bus image of the bus component, and an inverter image of the inverter component;

[0010] The audio data is vibration data of the inverter component, and the video data is a voltage transformation environment video of the inverter component;

[0011] According to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image, the photovoltaic components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle;

[0012] According to the fusion processing of the confluence data and the confluence image, the confluence components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle;

[0013] The inverter components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle according to the fusion processing of the inverter data, heat dissipation data, oil level data, the inverter image, the vibration data, and the transformer environment video.

[0014] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a photovoltaic power station operation and maintenance system based on multimodal data fusion, the photovoltaic power station operation and maintenance system based on multimodal data fusion comprising: a memory, a processor, a communication bus and a control program stored in the memory:

[0015] The communication bus is used to realize the connection and communication between the processor and the memory;

[0016] The processor is used to execute the control program to implement the steps of the photovoltaic power station operation and maintenance method based on multimodal data fusion as described above.

[0017] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a medium, which is a readable storage medium, on which a control program is stored, and when the control program is executed by a processor, the steps of the photovoltaic power station operation and maintenance method based on multimodal data fusion as described above are implemented.

[0018] The present invention is based on a photovoltaic power station operation and maintenance method, system and medium based on multimodal data fusion. The photovoltaic power station includes photovoltaic components, bus components, inverter components and detection components. An operation and maintenance cycle for regular operation and maintenance is pre-set. Multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle is received. The multimodal data includes text data, image data, audio data and video data that have been converted in format; specifically, the text data includes photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic components, bus data of the bus components, and inverter data, heat dissipation data and oil level data of the inverter components; the image data includes photovoltaic images of the photovoltaic components. , the confluence image of the confluence component, and the inverter image of the inverter component; the audio data is the vibration data of the inverter component, and the video data is the transformer environment video of the inverter component; and then according to the fusion processing of photovoltaic power generation data, photovoltaic temperature difference data and photovoltaic images, the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained; and according to the fusion processing of confluence data and confluence images, the confluence components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained; and according to the fusion processing of inverter data, heat dissipation data, oil level data, inverter images, vibration data, and transformer environment video, the inverter components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained. In this way, by collecting multimodal data including text data, image data, audio data and video data that reflect the operating status of the photovoltaic power station, and fusing at least two of the multimodal data, comprehensive and accurate operation and maintenance of photovoltaic components, confluence components and inverter components in the photovoltaic power station can be achieved. At the same time, because multimodal data is converted into a unified data format, the tedious processing of data in different formats can be avoided, making the fusion processing of various data more convenient, which is conducive to fast, comprehensive and accurate operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a first embodiment of a photovoltaic power station operation and maintenance method based on multimodal data fusion according to the present invention;

[0020] Figure 2 It is a flow chart of a second embodiment of a photovoltaic power station operation and maintenance method based on multimodal data fusion according to the present invention;

[0021] Figure 3 It is a flow chart of a third embodiment of a photovoltaic power station operation and maintenance method based on multimodal data fusion according to the present invention;

[0022] Figure 4 It is a structural schematic diagram of the hardware operating environment involved in an embodiment of a photovoltaic power station operation and maintenance system based on multimodal data fusion of the present invention.

[0023] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0025] The present invention provides a photovoltaic power station operation and maintenance method based on multimodal data fusion, please refer to Figure 1 , Figure 1 It is a flow chart of a first embodiment of a photovoltaic power station operation and maintenance method based on multimodal data fusion according to the present invention.

[0026] The embodiment of the present invention provides an embodiment of a photovoltaic power station operation and maintenance method based on multimodal data fusion. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that here. Specifically, the photovoltaic power station in this embodiment includes a photovoltaic component, a bus component, an inverter component, and a detection component, wherein the photovoltaic power station operation and maintenance method includes:

[0027] Step S10, obtaining multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, wherein the multimodal data includes text data, image data, audio data and video data.

[0028] The photovoltaic power station operation and maintenance method based on multimodal data fusion of this embodiment is applied to the server of the photovoltaic power station. The photovoltaic power station includes a photovoltaic assembly composed of multiple photovoltaic panels, a bus assembly composed of a bus cabinet, a DC distribution cabinet, etc., an inverter assembly composed of an inverter, an AC distribution cabinet, a transformer, etc., and a detection assembly for detecting photovoltaic assemblies, bus assemblies and inverter assemblies. The detection assembly includes at least temperature sensors, ammeters, voltmeters, infrared, drones, power robots and other equipment, so as to collect the appearance and operating performance parameters of photovoltaic assemblies, bus assemblies and inverter assemblies in the form of multimodal data such as images, audio, video and text through such equipment. The server of the photovoltaic power station supports communication connection with the detection assembly in the photovoltaic power station, so as to monitor the operating status of the photovoltaic power station through the fusion processing of the multimodal data detected by the detection assembly, and then realize the operation and maintenance of the photovoltaic power station.

[0029] Specifically, an operation and maintenance cycle is pre-set, such as one week, half a month, one month, one quarter, half a year, one year, etc. Whenever the operation and maintenance cycle ends, the server acquires the multimodal data of the photovoltaic power station collected by the detection component during the current operation and maintenance cycle. The acquisition method can be that the server sends a transmission request to each device in the detection component, and each device sends the multimodal data collected during the current operation and maintenance cycle to the server after receiving the transmission request. It can also be that each device in the detection component transmits the collected multimodal data to the storage unit of the server in real time, and the server retrieves the storage unit to achieve acquisition. The acquired multimodal data includes at least text data, image data, audio data and video data. Furthermore, corresponding to the photovoltaic components, busbar components and inverter components in the photovoltaic power station, the acquired text data at least includes the photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic components, the busbar data of the busbar components, the inversion data, heat dissipation data and oil level data of the inverter components, etc., which are used to reflect the operating status of the photovoltaic components, busbar components and inverter components; the acquired image data at least includes the photovoltaic image of the photovoltaic components, the busbar image of the busbar component, the inversion image of the inverter component, etc., which are used to reflect the appearance status data of the photovoltaic components, busbar components and inverter components; the acquired audio data and video data mainly include vibration data reflecting the operating vibration and operating environment of the inverter components and the transformer environment video.

[0030] Step S20, performing operation and maintenance of the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle according to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image.

[0031] Understandably, for photovoltaic components, the collected multimodal data includes photovoltaic power generation data, photovoltaic temperature difference data, and photovoltaic images. Therefore, in order to fully and accurately reflect its operating status, the photovoltaic power generation data, photovoltaic temperature difference data, and photovoltaic images related to the photovoltaic components can be fused and processed, and based on the results of the fusion processing, the photovoltaic components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle. Specifically, the steps of operating and maintaining the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle based on the fusion processing of the photovoltaic power generation data, photovoltaic temperature difference data, and the photovoltaic images include:

[0032] Step S21, grayscale processing is performed on the photovoltaic image, and a grayscale mean corresponding to each photovoltaic panel in the photovoltaic assembly is formed;

[0033] Step S22, forming matrix rows based on the monitoring period of the current operation and maintenance cycle, and forming matrix columns based on each of the photovoltaic panels;

[0034] Step S23, based on the matrix rows and the matrix columns, the photovoltaic power generation data, the photovoltaic temperature difference data and the grayscale mean are respectively formed into a power generation characteristic matrix, a temperature difference characteristic matrix and a grayscale characteristic matrix;

[0035] Step S24, performing fusion calculation on the power generation characteristic matrix, the temperature difference characteristic matrix and the grayscale characteristic matrix to obtain panel characteristic parameters of each photovoltaic panel in each monitoring period in the current operation and maintenance cycle;

[0036] Step S25, performing operation and maintenance of the current operation and maintenance cycle on each photovoltaic panel in the photovoltaic assembly according to the characteristic parameters of each panel.

[0037] Furthermore, in order to accurately reflect the operating status of each component in the photovoltaic power station during the operation and maintenance cycle, the operation and maintenance cycle can be divided into multiple monitoring periods, and the operating data of other components in the photovoltaic power station can be collected by detecting the components at each interval of the monitoring period. In addition, the operating data collected during the continuous monitoring period can be used to reflect the changes in the operating status of the components. In this embodiment, the multimodal data of each photovoltaic panel in each monitoring period of the operation and maintenance cycle can be used to reflect the changes in the operating status of each photovoltaic panel in the photovoltaic assembly, thereby realizing accurate operation and maintenance of each photovoltaic panel in the photovoltaic assembly.

[0038] Specifically, the collected photovoltaic power generation data is the power generation data of each photovoltaic panel in each monitoring period, the photovoltaic temperature difference data is the temperature difference data of at least two points selected for each photovoltaic panel in each monitoring period, and the photovoltaic image is the image taken of each photovoltaic panel that constitutes the photovoltaic assembly in each monitoring period.

[0039] Furthermore, the photovoltaic images of each photovoltaic panel are gray-scale processed to obtain the photovoltaic gray-scale images of each photovoltaic panel in each monitoring period. Then, the pixels in each gray-scale image are averaged to obtain the gray-scale mean of each photovoltaic panel in each monitoring period. In addition, the monitoring period of the current operation and maintenance cycle is formed into a matrix row, and each photovoltaic panel is formed into a matrix column, and then the photovoltaic power generation data of each photovoltaic panel in each monitoring period is filled based on the matrix rows and matrix columns to form a power generation characteristic matrix; and the photovoltaic temperature difference data of each photovoltaic panel in each monitoring period is filled based on the matrix rows and matrix columns to form a temperature difference characteristic matrix; and the gray-scale mean of each photovoltaic panel in each monitoring period is filled based on the matrix rows and matrix columns to form a gray-scale characteristic matrix.

[0040] In a specific embodiment, the number of photovoltaic panels of the photovoltaic assembly is 50, and the monitoring period of the operation and maintenance cycle is intervals of time A, which contains 50 intervals in total. Then the generated power generation characteristic matrix is:

[0041]

[0042] in, Indicates the photovoltaic power generation data of the first photovoltaic panel in monitoring period A, 2A, 3A...50A in the current operation and maintenance cycle; It represents the photovoltaic power generation data of the 1st, 2nd, 3rd, ..., 5th photovoltaic panels in the monitoring period A of the current operation and maintenance cycle. The temperature difference feature matrix and the gray feature matrix are also generated in this way, which will not be described in detail here.

[0043] Furthermore, in order to reflect the changes in the operating status of the photovoltaic panels during the operation and maintenance cycle, a formula is pre-set for calculating the panel characteristic parameters of the photovoltaic panels in each monitoring period of the current operation and maintenance cycle. The calculation formula is used to fuse the power generation characteristic matrix, the temperature difference characteristic matrix and the grayscale characteristic matrix to obtain the panel characteristic parameters of each photovoltaic panel in each monitoring period of the current operation and maintenance cycle. The calculation formula is specifically shown in the following formula (1):

[0044] (1);

[0045] Among them, P (i, t) represents the panel characteristic parameters of the i-th photovoltaic panel in the monitoring period t, Mit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, Nit represents the matrix value of the i-th column and the t-th row in the temperature difference characteristic matrix, Sit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, f1 (t ± 1) is the comparison function based on the monitoring period, and f2 (i ± 1) is the comparison function based on the photovoltaic panel.

[0046] Furthermore, the "|" in formula (1) is a comparison symbol. After the data of the same matrix row and the same matrix column in the power generation feature matrix, the temperature difference feature matrix and the grayscale feature matrix are multiplied to obtain the power generation feature, temperature difference feature and grayscale feature of the photovoltaic panel represented by the matrix row in the monitoring period represented by the matrix column, the f1 (t ± 1) function and the f2 (t ± 1) function are started by the comparison symbol. The above-mentioned feature of the photovoltaic panel is compared with its own features in the previous monitoring period and the next monitoring period by the f1 (t ± 1) function to obtain the first parameter representing the change of the above-mentioned feature of the photovoltaic panel in the continuous period, and the above-mentioned feature of the photovoltaic panel is compared with the features of the adjacent previous photovoltaic panel and the next photovoltaic panel in the same monitoring period by the f2 (i ± 1) function to obtain the second parameter representing the change of the above-mentioned feature of the photovoltaic panel relative to the adjacent photovoltaic panel. The first parameter and the second parameter form the panel feature parameter of the photovoltaic panel in a certain monitoring period in the current operation and maintenance cycle.

[0047] Understandably, for each photovoltaic panel in the photovoltaic assembly, the operating performance in the same monitoring period should not differ much; and for the same photovoltaic panel, the operating performance in the previous and subsequent continuous monitoring periods should also not differ much. Therefore, after obtaining the panel characteristic parameters of each photovoltaic panel in each monitoring period in the current operation and maintenance cycle, the first parameters of each photovoltaic panel in each monitoring period are compared with each other to find out whether there are first parameters with large differences; or each first parameter is first generated as a mean, and then each first parameter is compared with the mean to determine whether there are first parameters with large differences. If there are first parameters with large differences, the first parameter reflects that the photovoltaic panel has an abnormal operation in its corresponding monitoring period. Similarly, the second parameters of each photovoltaic panel in each monitoring period in the current operation and maintenance cycle are compared with each other to find out whether there are second parameters with large differences; or each second parameter is first generated as a mean, and then each second parameter is compared with the mean to determine whether there are second parameters with large differences. If there are second parameters with large differences, it reflects that the photovoltaic panel corresponding to the second parameter has an abnormal operation in the comparison monitoring period. Then, the operation and maintenance information of the abnormal photovoltaic panels is output for operation and maintenance, and the operation and maintenance of the current operation and maintenance cycle of each photovoltaic panel constituting the photovoltaic assembly is realized through the acquired panel characteristic parameters.

[0048] Step S30: performing operation and maintenance of the current operation and maintenance cycle on the confluence components in the photovoltaic power station according to the fusion processing of the confluence data and the confluence image.

[0049] Furthermore, for the busbar assembly, since it includes a busbar cabinet and a DC distribution cabinet, the multimodal data collected including busbar data and busbar images can be divided into busbar data and busbar images corresponding to the busbar cabinet, and busbar data and busbar images corresponding to the DC distribution cabinet. In order to reflect the operating status of the busbar cabinet and the DC distribution cabinet in the busbar assembly, the busbar data and busbar images related to the busbar cabinet can be fused, and the return data and busbar images related to the DC distribution cabinet can be fused. Then, according to the results of the respective fusion processing, the busbar cabinet and the DC distribution cabinet included in the busbar assembly in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle. Specifically, for the busbar cabinet in the busbar assembly, the step of operating and maintaining the busbar assembly in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the busbar data and the busbar image includes:

[0050] Step S31, generating a confluence parameter of each confluence node in the confluence component according to the photovoltaic power generation data and the confluence data;

[0051] Step S32, performing image feature calculation on each of the confluence images based on a preset image processing model to obtain a spatial region matrix of each of the confluence images;

[0052] Step S33, generating element matrices corresponding to each of the confluence nodes based on the similarities between the spatial region matrices;

[0053] Step S34, searching for a reference element matrix corresponding to each of the confluence nodes, and fusing each of the confluence parameters, each of the element matrices and each of the reference element matrices to obtain confluence characteristic parameters of each of the confluence nodes;

[0054] Step S35, performing operation and maintenance of the current operation and maintenance cycle on each confluence node in the confluence component according to each of the confluence characteristic parameters.

[0055] It can be understood that the busbar in the busbar assembly is used to converge the power generation of the photovoltaic panels in the photovoltaic assembly, and one photovoltaic panel corresponds to a busbar node in the busbar assembly, and the convergence amount of each busbar node should not be much different from the power generation of the corresponding photovoltaic panel. Therefore, it is possible to determine whether each busbar node in the busbar assembly can converge normally based on the power generation of each photovoltaic panel reflected in the photovoltaic power generation data and the convergence amount of each busbar node in the convergence data. Specifically, for each busbar node, the corresponding convergence amount is found from the collected convergence data, and the photovoltaic panel corresponding to the convergence node is found, and then the power generation corresponding to the photovoltaic panel is found from the photovoltaic power generation data, and the found convergence amount and power generation are formed as the power difference.

[0056] Furthermore, a mapping relationship between multiple difference intervals and parameters is pre-set, and the generated power difference is compared with each difference interval to determine the difference interval where the power difference is located, and then the parameter corresponding to the difference interval is searched according to the mapping relationship, which is the confluence parameter of the confluence node in the confluence component, reflecting the confluence performance of the confluence node. In one embodiment, the smaller the numerical range of the difference interval, the smaller the mapped parameter value, indicating that the difference between the power generation and the return flow is not large, and the confluence node has a good confluence performance.

[0057] Furthermore, a graph processing model is pre-set, and the image features of each collected confluence image are calculated by the preset graph processing model to obtain a spatial region matrix of each confluence image, and the spatial region matrix reflects the spatial position of the object in the confluence image and the features of the object at each spatial position. The specific calculation formula is shown in the following formula (2).

[0058] (2);

[0059] Among them, Rj represents the spatial area matrix of the j-th confluence image, Hj represents the length of the j-th confluence image, Wj represents the width of the j-th confluence image, G2 represents the coordinate conversion function, G1 represents the feature extraction function, K1 represents the number of convolution kernels of the preset image processing model, k2 represents the number of steps of the preset image processing model, k3 represents the number of channels of the preset image processing model, and T* represents the formation of the converted coordinates and the extracted features into a matrix.

[0060] In a specific embodiment, "||" is used to indicate the value setting of k1, k2 and k3, for example, the number of convolution kernels k1=3*3, the number of steps k2=1, and the number of channels k3=3. A certain confluence image is processed by a preset image processing model, and the pixel coordinates of the object pixels therein are converted to the representation coordinate system by the G2 coordinate conversion function, respectively, as {(x1, y1), (x2, y2), (x3, y3) ... (xn, yn)}, and the features corresponding to each pixel in advance by the G2 feature extraction function are {(r1, g1, b1), (r2, g2, b2), (r3, g3, b3) ... (rn, gn, bn)}, and then the extracted pixel coordinates and features are formed into a matrix according to the corresponding relationship. For example, the corresponding coordinates and features are formed into the same matrix row, that is, x1, y1, r1, g1, b1 are formed into the first row of the matrix, x2, y2, r2, g2, b2 are formed into the second row of the matrix, and so on, until all pixel coordinates and features are formed into matrix rows. Of course, the matrix can also be formed in a column manner, that is, the corresponding coordinates and features are formed into the same matrix column, until all pixel coordinates and features are formed into matrix columns.

[0061] Understandably, there may be multiple converged images among the collected converged images representing the same object, that is, the images of a certain object are collected at different angles to obtain multiple converged images of the object. The pixel coordinates and pixel features of such converged images are similar, and the formed spatial region matrix is ​​also similar because it reflects the spatial position of the same object and the features of the object in each spatial position. Therefore, the similarity calculation can be performed on each spatial region matrix to determine the target spatial region matrix with high similarity in each spatial region matrix. This type of target spatial region matrix reflects the same object element of the converged node, so it can be used as the element matrix of the converged node.

[0062] In addition, for the items of the confluence node, reference pictures representing their normal state are pre-set, such as pictures of confluence terminals with clean and tidy appearance, no rust, and no damage, and such pictures are pre-generated as reference element matrices. After obtaining the element matrix of each confluence node, the reference element matrix corresponding to each confluence node is searched, and the reference element matrix found is fused together with each confluence parameter and each element matrix to obtain the confluence characteristic parameters of each confluence node. Among them, the fusion process can be to calculate the similarity between the reference element matrix and the element matrix of each confluence node, obtain the similarity value between the two, and correct the calculated similarity value in combination with the confluence parameter to obtain the confluence characteristic parameter. For example, if the calculated similarity value is high, it means that the actual state of the confluence node is not much different from the theoretical reference state. In this case, the confluence performance of the confluence node should be good, so the confluence performance reflected by the confluence parameter can be used to correct the similarity value to obtain the confluence characteristic parameter that finally reflects the confluence performance of the confluence node. For example, in the case where the similarity value is high, if the convergence performance reflected by the convergence parameter is indeed good, the similarity value can be corrected to be higher to obtain the convergence characteristic parameter reflecting the better convergence performance. Conversely, if the convergence performance reflected by the convergence parameter is poor, the similarity value can be corrected to be lower to obtain the convergence characteristic parameter reflecting the poor convergence performance.

[0063] Furthermore, after obtaining the convergence characteristic parameters of each convergence node in the convergence component, each convergence node can be operated and maintained according to each convergence characteristic parameter. Specifically, a parameter indicating normal convergence performance can be set, and each convergence characteristic parameter can be compared with the parameter; or each convergence characteristic parameter can be averaged to obtain a mean calculation result, and each convergence characteristic parameter can be compared with the mean calculation result. Among them, for convergence characteristic parameters that are less than the parameter or less than the mean calculation result, the convergence performance of the corresponding convergence node is poor and requires operation and maintenance. Therefore, the corresponding operation and maintenance information can be output for this type of convergence node to perform operation and maintenance on each convergence node in the convergence component during the current operation and maintenance cycle.

[0064] Step S40, performing operation and maintenance of the inverter components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the inverter data, heat dissipation data, oil level data, the inverter image, the vibration data, and the transformer environment video.

[0065] It can be understood that for the inverter components in the photovoltaic power station, the collected multimodal data include text-based inverter data, heat dissipation data, oil level data, image-based inverter images, audio-based vibration data, and video-based transformer environment videos. Therefore, in order to fully and accurately reflect its operating status, the inverter data, heat dissipation data, oil level data, inverter images, vibration data, and transformer environment videos related to the inverter components can be fused and processed, and based on the results of their respective fusion processing, the inverter components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle.

[0066] The photovoltaic power station operation and maintenance method based on multimodal data fusion implemented in this embodiment, the photovoltaic power station includes photovoltaic components, bus components, inverter components and detection components, and an operation and maintenance cycle for regular operation and maintenance is pre-set. The multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle is received, and the multimodal data includes format-converted text data, image data, audio data and video data; specifically, the text data includes photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic components, bus data of the bus components, and inverter data, heat dissipation data and oil level data of the inverter components; the image data includes photovoltaic images of the photovoltaic components, and the bus components include the image data of the inverter components. The confluence image of the flow component and the inverter image of the inverter component; the audio data is the vibration data of the inverter component, and the video data is the voltage-converting environment video of the inverter component; and then according to the fusion processing of photovoltaic power generation data, photovoltaic temperature difference data and photovoltaic images, the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained; and according to the fusion processing of confluence data and confluence images, the confluence components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained; and according to the fusion processing of inverter data, heat dissipation data, oil level data, inverter images, vibration data, and voltage-converting environment videos, the inverter components in the photovoltaic power station in the current operation and maintenance cycle are operated and maintained. In this way, by collecting multimodal data including text data, image data, audio data and video data reflecting the operating status of the photovoltaic power station, and fusing at least two of the multimodal data, comprehensive and accurate operation and maintenance of photovoltaic components, confluence components and inverter components in the photovoltaic power station can be achieved. At the same time, because multimodal data is converted into a unified data format, the tedious processing of data in different formats can be avoided, making the fusion processing of various data more convenient, which is conducive to fast, comprehensive and accurate operation and maintenance.

[0067] For further information, please refer to Figure 2 Based on the first embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion of the present invention, a second embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion of the present invention is proposed.

[0068] The second embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion is different from the first embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion in that the inverter assembly includes an inverter, a power distribution cabinet and a transformer;

[0069] The step of operating and maintaining the inverter components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the inverter data, the heat dissipation data, the oil level data, the inverter image, the vibration data, and the transformer environment video includes:

[0070] Step S41, performing operation and maintenance of the inverter in the current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data;

[0071] Step S42, performing operation and maintenance of the power distribution cabinet in the current operation and maintenance cycle according to the power distribution data in the inverter data, the power distribution and heat dissipation data in the heat dissipation data, and the fusion processing of the power distribution cabinet image in the inverter image;

[0072] Step S43, performing operation and maintenance on the transformer in the current operation and maintenance cycle according to the fusion processing of the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the transformer environment video.

[0073] It can be understood that the photovoltaic power station includes an inverter box for converting direct current into alternating current, a distribution cabinet for distributing electric energy, and a transformer for realizing voltage conversion. In this embodiment, the inverter, distribution cabinet and transformer are collectively regarded as an inverter component, and the inverter, distribution cabinet and transformer in the inverter component are operated and maintained separately.

[0074] Specifically, the collected inverter data includes inverter box data reflecting the inverter box's conversion of DC power, the heat dissipation data includes inverter heat dissipation data reflecting the inverter box's heat dissipation, and the inverter image includes an inverter box image reflecting whether the inverter box's appearance is intact. In addition, the collected vibration data is data reflecting the magnitude of the inverter box's vibration during operation. Therefore, based on the fusion processing of such inverter box data, inverter heat dissipation data, inverter box image, and vibration data, the inverter can be operated and maintained in the current operation and maintenance cycle.

[0075] Furthermore, the collected inverter data also includes distribution data reflecting whether the distribution cabinet operates normally in each monitoring period in the current operation and maintenance cycle, the heat dissipation data includes distribution heat dissipation data reflecting the heat dissipation of the distribution cabinet, and the inverter image includes a distribution cabinet image reflecting whether the appearance of the distribution cabinet is intact. Therefore, the operation and maintenance of the distribution cabinet in the current operation and maintenance cycle can be realized based on the fusion processing of such distribution data, distribution heat dissipation data, and distribution cabinet image. Specifically, through the distribution data in each monitoring period, it is determined whether the distribution cabinet operates normally in each monitoring period, and then the distribution parameters corresponding to each monitoring period are generated; and according to the similarity between the collected distribution cabinet image and the preset distribution cabinet reference image, it is determined whether the appearance of the distribution cabinet is intact, and then the corresponding image parameters are generated; and according to the comparison between the collected distribution heat dissipation data and the reference heat dissipation parameters corresponding to each monitoring period, the heat dissipation parameters of each monitoring period are generated. Afterwards, according to each monitoring period, the power distribution parameters, image parameters and heat dissipation parameters are integrated and processed to generate parameters indicating whether there are any abnormalities in the power distribution, appearance and heat dissipation of the distribution cabinet in each monitoring period. Then, based on the parameters, the power distribution cabinet is operated and maintained in the current operation and maintenance cycle in terms of power distribution, appearance and heat dissipation.

[0076] Furthermore, the collected inverter data also includes transformer data reflecting the voltage transformation of the transformer, and the inverter image includes a transformer image reflecting whether the transformer appearance is intact. In addition, the collected oil level data is the oil volume data contained in the oil level device in the transformer, and the transformer environment video is a video reflecting the environment in which the transformer is located. Therefore, based on the fusion processing of such transformer data, transformer image, oil level data, and transformer environment video, the operation and maintenance of the transformer in the current operation and maintenance cycle can be realized.

[0077] The step of performing operation and maintenance of the inverter in the current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data includes:

[0078] Step S411, generating the inverter box data into inverter efficiency parameters for each monitoring period in the current operation and maintenance cycle;

[0079] Step S412, obtaining weather data in the current operation and maintenance cycle, and generating heat dissipation parameters of the inverter box in each monitoring period in the current operation and maintenance cycle according to the weather data and the inverter heat dissipation data;

[0080] Step S413, comparing the inverter box image with a preset inverter reference image to generate inverter image parameters of the inverter box;

[0081] Step S414, processing the vibration data based on a preset vibration algorithm to generate a vibration amplitude parameter of the inverter box in the current operation and maintenance cycle;

[0082] Step S415, the inverter efficiency parameter, heat dissipation parameter, inverter image parameter and vibration amplitude parameter are integrated based on each of the monitoring time periods to obtain inverter characteristic parameters, and the inverter is operated and maintained in the current operation and maintenance cycle according to each of the inverter characteristic parameters.

[0083] Furthermore, corresponding to each monitoring period in the current operation and maintenance cycle, the inverter box data includes input power data and output power data of each monitoring period, and such output power data and input power data are ratio-operated to generate inverter efficiency parameters of each monitoring period in the current operation and maintenance cycle.

[0084] It is understandable that the heat dissipation performance of the inverter is not only related to the performance of the heat dissipation device equipped with the inverter, but also to the weather of the environment in which the inverter is located. Therefore, the weather data of the inverter in the current operation and maintenance cycle is obtained, and the weather data obtained may include temperature, humidity, wind speed, etc. The heat dissipation performance of the heat dissipation device under different weather conditions is different. The heat dissipation representation parameter pair of the heat dissipation device under each weather state is pre-set. The heat dissipation representation parameter pair includes at least one weather parameter and one representation parameter, and the representation parameter reflects the heat dissipation performance of the heat dissipation device under the weather parameter. After obtaining the weather data of each monitoring period in the current operation and maintenance cycle, the heat dissipation representation parameter pair corresponding to each weather data and the representation parameters in each heat dissipation representation parameter pair are searched. Then, the heat dissipation data of each monitoring period contained in the inverter heat dissipation data is compared with each representation parameter to generate the heat dissipation parameters of the inverter box in each monitoring period in the current operation and maintenance cycle. The heat dissipation parameter reflects the abnormality of the heat dissipation data of the time period. If the difference between the heat dissipation parameter of the time period and the representation parameter is large after comparison, it means that the heat dissipation parameter of the time period is abnormal, so a heat dissipation parameter representing the abnormality is generated, otherwise a heat dissipation parameter representing the abnormality is generated. In addition, the size of the heat dissipation parameter can also be used to represent the size of the abnormality. For example, the larger the heat dissipation parameter value is, the greater the abnormality is.

[0085] Furthermore, in order to monitor whether the appearance of the inverter box is damaged, an image indicating that the inverter box is intact is pre-set as a preset inverter reference image, and the captured inverter box image is compared with the preset reference image to generate the image similarity between the two. In addition, a preset similarity value indicating the magnitude of the similarity is pre-set, and the generated image similarity is compared with the preset similarity value to determine whether the image similarity is greater than or equal to the preset similarity value. If it is greater than or equal to, it means that the similarity between the two is high, and the appearance of the inverter box can be determined to be intact, and an inverter image parameter indicating that it is intact is generated. On the contrary, if it is determined through comparison that the image similarity value is less than the preset similarity value, it means that the similarity between the two is low, and the appearance of the inverter box can be determined to be damaged, and the corresponding inverter image parameter is generated according to the degree of damage to the appearance of the inverter box. Among them, the degree of damage to the appearance of the inverter box can be reflected by the degree to which the image similarity value is less than the preset similarity value.

[0086] Furthermore, for the vibration data reflecting the vibration magnitude generated during the operation of the inverter box, a preset vibration algorithm is pre-set, and the vibration data is processed by the preset vibration algorithm to generate a vibration amplitude parameter reflecting the vibration magnitude of the inverter box in each monitoring period of the current operation and maintenance cycle. The calculation formula of the preset vibration algorithm is shown in the following formula (3).

[0087] (3);

[0088] Wherein, H(A, t) represents the vibration amplitude parameter A within the monitoring period t, k represents the kth vibration data of the kth monitoring period, represents the extraction coefficient of the kth vibration data, Represents the amplitude extraction function. Extraction coefficient It is related to the ground vibration condition in the kth monitoring period where the kth vibration data is located, indicating the impact of the ground vibration on the operating vibration of the inverter box.

[0089] Furthermore, the generated inverter efficiency parameters, heat dissipation parameters, inverter image parameters and vibration amplitude parameters are fused according to the respective monitoring periods. Specifically, each inverter efficiency parameter, heat dissipation parameter, inverter image parameter and vibration amplitude parameter carries a time period tag representing the corresponding monitoring period. According to the time period tags carried by each, the inverter efficiency parameters, heat dissipation parameters, inverter image parameters and vibration amplitude parameters from the same monitoring period are fused into a data group, and then the inverter characteristic parameters are generated according to the overall abnormal conditions of the inverter reflected by each data in the data group. Among them, the weight value can be pre-set according to the influence of the inverter efficiency, heat dissipation, inverter appearance and vibration on the inverter conversion, and the inverter efficiency parameter, heat dissipation parameter, inverter image parameter and vibration amplitude parameter in the data group are weighted and fused according to the weight value to obtain the inverter characteristic parameters.

[0090] Furthermore, the inverter is operated and maintained in the current operation and maintenance cycle according to the inverter characteristic parameters of each monitoring period. Specifically, it is determined whether the change state of each inverter characteristic parameter is stable. If it is stable, it means that the operation state of the inverter is normal. If it is not stable, the inverter characteristic parameter that begins to fluctuate is determined, and the inverter efficiency parameter, heat dissipation parameter, image parameter and vibration amplitude parameter that generate the fluctuating inverter characteristic parameter are found. Thereafter, based on the inverter performance reflected by the inverter efficiency parameter, heat dissipation parameter, image parameter and vibration amplitude parameter, it is determined whether there is one or more of the conversion abnormality, heat dissipation abnormality, appearance abnormality and vibration abnormality, and then according to the operation and maintenance information corresponding to the determined abnormal output, the inverter is subjected to targeted and comprehensive operation and maintenance in the current operation and maintenance cycle.

[0091] Furthermore, the step of operating and maintaining the transformer in the current operation and maintenance cycle according to the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the fusion processing of the transformer environment video includes:

[0092] Step S431, generating the voltage transformation data and the oil level data into voltage transformation efficiency parameters and oil level change parameters of each monitoring period in the current operation and maintenance cycle respectively;

[0093] Step S432, comparing the transformer image with a preset transformer reference image to generate transformer image parameters of the transformer;

[0094] Step S433, extracting the to-be-recognized images corresponding to each of the monitoring time periods from the transformer environment video, and extracting the environmental characteristic parameters corresponding to the transformer from the to-be-recognized images based on a preset extraction model;

[0095] Step S434, the transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter are integrated based on each of the monitoring time periods to obtain transformer characteristic parameters, and the transformer is operated and maintained in the current operation and maintenance cycle according to each of the transformer characteristic parameters.

[0096] Furthermore, corresponding to each monitoring period in the current operation and maintenance cycle, the transformer data includes input power data and output power data of each monitoring period, and the ratio operation of such output power data and input power data is performed to generate transformer efficiency parameters of each monitoring period in the current operation and maintenance cycle. In addition, the oil level data also includes the oil level data at the beginning of each monitoring period and the oil level data at the end of each period, and the oil level change data of each monitoring period in the current operation and maintenance cycle is obtained by calculating the difference between the oil level data at the beginning of the period and the oil level data at the end of the period.

[0097] In addition, similar to the inverter box, in order to monitor whether the transformer appearance is damaged, an image indicating that the transformer is intact is pre-set as a preset transformer reference image, and the collected transformer image is compared with the preset transformer reference image to generate the image similarity between the two. The image similarity is compared with the preset similarity value to determine whether the image similarity is greater than or equal to the preset similarity value. If it is greater than or equal to, it means that the similarity between the two is high, and the transformer appearance can be determined to be intact, and the transformer image parameters representing integrity are generated. On the contrary, if it is determined through comparison that the image similarity is less than the preset similarity value, it means that the similarity between the two is low, and the transformer appearance can be determined to be damaged, and the corresponding transformer image parameters are generated according to the degree of damage to the transformer appearance. Among them, the degree of damage to the transformer appearance can be reflected by the degree to which the image similarity is less than the preset similarity value.

[0098] Furthermore, for the transformer environment video reflecting the environment in which the transformer is located, frame extraction processing is first performed based on each monitoring period. Specifically, the transformer environment video is first divided into video segments corresponding to each monitoring period according to the generation time of the transformer environment video. In addition, a frame extraction time interval is pre-set, and frame extraction processing is performed for each video segment according to the frame extraction time interval to obtain multiple image frames as the images to be identified corresponding to the monitoring period.

[0099] Furthermore, a preset extraction model is pre-set, and a feature extraction operation is performed on the image to be identified through an extraction formula in the preset extraction model, and environmental feature parameters corresponding to the transformer are extracted from the image to be identified. The extraction formula in the preset extraction model is shown in the following formula (4).

[0100] ; (4)

[0101] Among them, px is the environmental feature parameter corresponding to the xth image to be identified. is the activation function of the preset extraction model, m is the connection index between the xth image to be recognized and the zth layer neuron of the preset extraction model, is the weight parameter of the neurons in the zth layer of the preset extraction model, is the RGB feature value of each pixel in the x-th image to be identified, and bz is the bias value of the z-th layer neuron of the preset extraction model.

[0102] Furthermore, the generated transformer efficiency parameters, oil level change parameters, transformer image parameters and environmental characteristic parameters are fused according to the monitoring periods in which they are located. Specifically, each transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter carries a time period label indicating the corresponding monitoring period. According to the time period labels carried by each, the transformer efficiency parameters, oil level change parameters, change image parameters and environmental characteristic parameters from the same monitoring period are fused into a data group, and then the transformer characteristic parameters are generated according to the overall abnormal conditions of the transformer reflected by each data in the data group. Among them, the weight value can be pre-set according to the influence of the transformer efficiency, oil level, transformer appearance and environment on the transformer transformation, and the transformer efficiency parameters, oil level change parameters, transformer image parameters and environmental characteristic parameters in the data group are weighted and fused according to the weight value to obtain the transformer characteristic parameters.

[0103] Furthermore, the transformer is operated and maintained in the current operation and maintenance cycle according to the transformer characteristic parameters of each monitoring period. Specifically, it is determined whether the change state of each transformer characteristic parameter is stable. If it is stable, it means that the operation state of the transformer is normal. If it is not stable, the transformer characteristic parameter that begins to fluctuate is determined, and the transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter that generate the fluctuating transformer characteristic parameter are found. Thereafter, based on the transformer performance reflected by the transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter, it is determined whether there is one or more of transformer abnormality, heat dissipation abnormality, appearance abnormality and vibration abnormality, and then the operation and maintenance information corresponding to the determined abnormal output is used to realize the targeted and comprehensive operation and maintenance of the transformer in the current operation and maintenance cycle.

[0104] This embodiment performs operation and maintenance on the inverter, power distribution cabinet, and transformer in the inverter assembly respectively. For the inverter, operation and maintenance are performed based on the fusion processing of the inverter box data, heat dissipation data, inverter box image, and vibration data that reflect its inverter performance; for the power distribution cabinet, operation and maintenance are performed based on the fusion processing of the power distribution data, heat dissipation data, and inverter box image that reflect its power distribution performance; for the transformer, operation and maintenance are performed based on the fusion processing of the transformer data, oil level data, transformer image, and transformer environment video that reflect its transformer performance. In this way, the operation and maintenance of the inverter, power distribution cabinet, and transformer comprehensively considers the multi-dimensional factors related to their respective operations, and improves the accuracy of the operation and maintenance of the inverter, power distribution cabinet, and transformer.

[0105] For further information, please refer to Figure 3 Based on the first and second embodiments of the photovoltaic power station operation and maintenance method based on multimodal data fusion of the present invention, a third embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion of the present invention is proposed.

[0106] The third embodiment of the photovoltaic power station operation and maintenance method based on multimodal data fusion is different from the first and second embodiments of the photovoltaic power station operation and maintenance method based on multimodal data fusion in that the step of obtaining the multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle includes:

[0107] Step S11, receiving text data, image data, audio data and video data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, and determining whether the text data, image data, audio data and video data all have complete data packet numbers;

[0108] Step S12: if all of the data packets have complete data packet numbers, format conversion is performed on the text data, image data, audio data and video data to form the multimodal data;

[0109] Step S13, if there is data to be transmitted that does not have a complete data packet number among the text data, image data, audio data and video data, a transmission request is sent to the detection component based on the missing data packet number of the data to be transmitted to obtain a complete data packet to form the multimodal data.

[0110] Understandably, the graphic data and audio and video data collected by the detection component may be at risk of leakage. To this end, the present embodiment is provided with a partial data encryption transmission mechanism between the detection component and the data transmission of the server, so as to avoid the complex encryption and decryption problems caused by encrypting all data while ensuring the security of the transmitted data. Specifically, before the detection component transmits text data, image data, audio data and video data to the server, the number of data packets formed by each type of data for transmission is transmitted as one of the data to be transmitted. After receiving the text data, image data, audio data and video data transmitted by the detection component, the server detects each type of data according to the number of data packets carried in each type of data, and determines whether there is a complete data packet number. Among them, the data packet number is a sequential numbering based on the number of data packets. If the number of data packets is 10, the data packet number is numbered from 1 to 10 in sequence.

[0111] Further, if it is determined that the data packet number of the text data, image data, audio data and video data is complete relative to the number of data packets, it means that the detection component has sent all the text data, image data, audio data and video data collected in the current operation and maintenance cycle to the server. At this time, the server can directly perform the operation and maintenance of the current operation and maintenance cycle based on this type of data. However, it should be noted that this type of text data, image data, audio data and video data are collected by different devices that constitute the detection component, and different devices use different communication protocols, so that the transmitted data also has different formats. In order to facilitate unified fusion processing, the server is pre-set with a format conversion mechanism, setting a unified data format and a conversion algorithm between the unified data format and other data formats. For the received text data, image data, audio data and video data, first determine whether the format of this type of data is the same as the unified data format. If the same, no conversion is required, and it can be directly used as multimodal data for fusion processing; if different, it is necessary to convert it according to the conversion algorithm between its format and the unified data format to obtain multimodal data with a unified format for fusion processing.

[0112] Furthermore, if it is determined that the data packet number of any one of the text data, image data, audio data and video data is incomplete, it means that the detection component has not transmitted all the data collected in the current operation and maintenance cycle to the server, and the missing data needs to be obtained by encryption and decryption to ensure the security of the transmitted data. The received data without a complete data packet is regarded as the data to be transmitted, and the missing data packet number in the data to be transmitted is searched, and then the found data packet number is formed into a transmission request to the detection component, so that the detection component can transmit the missing data with the data packet number to the server, so that the server can obtain a complete data packet to form multimodal data.

[0113] Furthermore, for the security of the transmitted data, the transmission request formed includes a preset encryption algorithm, so that the detection component encrypts the missing data packet according to the encryption algorithm and then transmits it to the server, and the server needs to perform a decryption operation after receiving the encrypted data packet. Specifically, the step of sending a transmission request to the detection component based on the data packet number missing from the data to be transmitted to obtain a complete data packet to form the multimodal data includes:

[0114] Step S14, receiving the encrypted data packet sent by the detection component based on the transmission request, and calculating the missing data packet number based on a preset decryption algorithm to obtain a decryption key;

[0115] Step S15, decrypting the encrypted data packet based on the decryption key to obtain a decrypted data packet, and merging the decrypted data packet with the data to be transmitted according to the missing data packet number to form a complete data packet;

[0116] Step S16, performing format conversion processing on the complete data packet to form the multimodal data.

[0117] Furthermore, in addition to the encryption algorithm, the server is also pre-set with a decryption algorithm. The encryption algorithm and the decryption algorithm are inverse operations of each other, and are both generated based on the missing data packet number, so that each time the data is transmitted, the missing data packet number is different, and different encryption keys and decryption keys are formed by the encryption and decryption algorithm for encryption and decryption, further ensuring the security of the transmitted data. After receiving the encrypted data packet generated by the detection component based on the transmission request, the server calculates the missing data packet code through the preset decryption algorithm to obtain the decryption key. The preset decryption algorithm is shown in the following formula (5).

[0118] (5);

[0119] Where S represents the decryption key, represents the data volume of the nth data packet, An represents the data volume of the complete data packet, x1 and x2 are the missing data packet numbers, and y1 and y2 are the adjacent data packet number pairs corresponding to the missing data packet numbers. When there are multiple missing data packet numbers, y1 and y2 are also multiple pairs. For example, x1 is 5 and x2 is 8, which means that when the 5th and 8th data packets are missing, the adjacent data packet number pairs y1 and y2 corresponding to x1 are 4 and 6 respectively, and the adjacent data packet number pairs y1 and y2 corresponding to x2 are 7 and 9 respectively.

[0120] Furthermore, after the decryption key is obtained through calculation, the received encrypted data packet can be interfaced with the decryption key to obtain the decrypted data packet, which is the data packet missing in the data to be transmitted. In addition, since the position of the missing data packet in the data packet to be transmitted can be reflected by the missing data packet number, the decrypted data packet can be added to the position of the data to be transmitted represented by the missing data packet number according to the missing data packet number, and merged with the data to be transmitted to form a complete data packet.

[0121] Similarly, because the format of the complete data packet may be different from the unified data format, the complete data needs to be converted. That is, first determine whether the format of the complete data packet is the same as the unified data format. If they are the same, no conversion is required and they can be directly used as multimodal data for fusion processing. If they are not the same, the format of the complete data packet is converted into a unified data format according to the conversion algorithm between the two formats, and the converted data is used as multimodal data for fusion processing.

[0122] This embodiment sets encryption processing for data with transmission security requirements. And encryption does not encrypt the complete data, but selects some data packets for encryption, so as to reduce the amount of data of the encryption and decryption algorithm while ensuring the security of data transmission. In addition, the encryption and decryption algorithm is calculated based on the missing data packet encoding, which is conducive to further ensuring the security of data transmission. In addition, through the format conversion mechanism, data with different formats transmitted based on different communication protocols are converted into a unified data format, forming multimodal data with the same data format, which is convenient for the fusion processing of multimodal data.

[0123] In addition, an embodiment of the present invention also provides a photovoltaic power station operation and maintenance system based on multimodal data fusion. The photovoltaic power station includes photovoltaic components, confluence components, inverter components, and detection components. Figure 4 , Figure 4 It is a structural schematic diagram of the equipment hardware operating environment involved in the embodiment of the photovoltaic power station operation and maintenance system based on multimodal data fusion of the present invention.

[0124] like Figure 4 As shown, the photovoltaic power station operation and maintenance system based on multimodal data fusion may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0125] Those skilled in the art will understand that Figure 4 The hardware structure of the photovoltaic power station operation and maintenance system based on multimodal data fusion shown in the figure does not constitute a limitation on the photovoltaic power station operation and maintenance system based on multimodal data fusion, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0126] like Figure 4As shown, the memory 1005 as a readable storage medium may include an operating system, a network communication module, a user interface module and a control program. The operating system is a program for managing and controlling the photovoltaic power station operation and maintenance system and software resources based on multimodal data fusion, and supports the operation of the network communication module, the user interface module, the control program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0127] exist Figure 4 In the hardware structure of the photovoltaic power station operation and maintenance system based on multimodal data fusion shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the control program stored in the memory 1005 and perform the following operations:

[0128] Acquire multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, wherein the multimodal data includes text data, image data, audio data, and video data that have been converted into a format; wherein:

[0129] The text data includes: photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic component, confluence data of the confluence component, inversion data, heat dissipation data and oil level data of the inversion component;

[0130] The image data includes: a photovoltaic image of the photovoltaic component, a bus image of the bus component, and an inverter image of the inverter component;

[0131] The audio data is vibration data of the inverter component, and the video data is a voltage transformation environment video of the inverter component;

[0132] According to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image, the photovoltaic components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle;

[0133] According to the fusion processing of the confluence data and the confluence image, the confluence components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle;

[0134] The inverter components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle according to the fusion processing of the inverter data, heat dissipation data, oil level data, the inverter image, the vibration data, and the transformer environment video.

[0135] Furthermore, the step of operating and maintaining the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle according to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image includes:

[0136] Performing grayscale processing on the photovoltaic image and forming a grayscale mean corresponding to each photovoltaic panel in the photovoltaic assembly;

[0137] Forming matrix rows based on the monitoring period of the current operation and maintenance cycle, and forming matrix columns based on each of the photovoltaic panels;

[0138] Based on the matrix rows and the matrix columns, the photovoltaic power generation data, the photovoltaic temperature difference data and the grayscale mean are respectively formed into a power generation characteristic matrix, a temperature difference characteristic matrix and a grayscale characteristic matrix;

[0139] The power generation characteristic matrix, the temperature difference characteristic matrix and the grayscale characteristic matrix are fused and calculated to obtain the panel characteristic parameters of each photovoltaic panel in each monitoring period in the current operation and maintenance cycle, wherein the calculation formula is:

[0140] ;

[0141] According to the characteristic parameters of each panel, each photovoltaic panel in the photovoltaic assembly is operated and maintained in the current operation and maintenance cycle;

[0142] Among them, P (i, t) represents the panel characteristic parameters of the i-th photovoltaic panel in the monitoring period t, Mit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, Nit represents the matrix value of the i-th column and the t-th row in the temperature difference characteristic matrix, Sit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, f1 (t ± 1) is the comparison function based on the monitoring period, and f2 (i ± 1) is the comparison function based on the photovoltaic panel.

[0143] Furthermore, the step of performing operation and maintenance of the confluence components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the confluence data and the confluence image includes:

[0144] Generate a confluence parameter of each confluence node in the confluence component according to the photovoltaic power generation data and the confluence data;

[0145] Based on the preset image processing model, image features of each confluence image are calculated to obtain a spatial region matrix of each confluence image. The calculation formula is:

[0146] ;

[0147] Wherein, Rj represents the spatial region matrix of the j-th confluence image, Hj represents the length of the j-th confluence image, Wj represents the width of the j-th confluence image, G2 represents the coordinate conversion function, G1 represents the feature extraction function, K1 represents the number of convolution kernels of the preset image processing model, k2 represents the number of steps of the preset image processing model, k3 represents the number of channels of the preset image processing model, and T* represents the formation of the converted coordinates and the extracted features into a matrix;

[0148] Based on the similarity between the spatial region matrices, generating element matrices corresponding to the confluence nodes respectively;

[0149] Finding the reference element matrix corresponding to each confluence node, and fusing each confluence parameter, each element matrix and each reference element matrix to obtain the confluence characteristic parameter of each confluence node;

[0150] According to each of the confluence characteristic parameters, each confluence node in the confluence component is operated and maintained in the current operation and maintenance cycle.

[0151] Furthermore, the inverter assembly includes an inverter, a power distribution cabinet and a transformer;

[0152] The step of operating and maintaining the inverter components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the inverter data, the heat dissipation data, the oil level data, the inverter image, the vibration data, and the transformer environment video includes:

[0153] Performing operation and maintenance of the inverter in a current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data;

[0154] According to the power distribution data in the inverter data, the power distribution and heat dissipation data in the heat dissipation data, and the fusion processing of the power distribution cabinet image in the inverter image, the power distribution cabinet is operated and maintained in the current operation and maintenance cycle;

[0155] The transformer is operated and maintained in the current operation and maintenance cycle according to the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the fusion processing of the transformer environment video.

[0156] Further, the step of performing operation and maintenance of the inverter in the current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data includes:

[0157] The inverter box data is generated as the inverter efficiency parameters of each monitoring period in the current operation and maintenance cycle;

[0158] Acquire weather data in the current operation and maintenance cycle, and generate heat dissipation parameters of the inverter box in each monitoring period in the current operation and maintenance cycle according to the weather data and the inverter heat dissipation data;

[0159] Comparing the inverter box image with a preset inverter reference image to generate inverter image parameters of the inverter box;

[0160] Based on a preset vibration algorithm, the vibration data is processed to generate the vibration amplitude parameter of the inverter box in the current operation and maintenance cycle. The calculation formula of the preset vibration algorithm is:

[0161] ;

[0162] Where H(A, t) represents the vibration amplitude parameter A within the monitoring period t, k represents the kth vibration data, represents the extraction coefficient of the kth vibration data, represents the amplitude extraction function;

[0163] The inverter efficiency parameter, heat dissipation parameter, inverter image parameter and vibration amplitude parameter are integrated based on each monitoring period to obtain the inverter characteristic parameter, and the inverter is operated and maintained in the current operation and maintenance cycle according to each inverter characteristic parameter.

[0164] Furthermore, the step of operating and maintaining the transformer in the current operation and maintenance cycle according to the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the fusion processing of the transformer environment video includes:

[0165] The voltage transformation data and the oil level data are respectively generated as voltage transformation efficiency parameters and oil level change parameters of each monitoring period in the current operation and maintenance cycle;

[0166] Comparing the transformer image with a preset transformer reference image to generate transformer image parameters of the transformer;

[0167] Extract the to-be-recognized images corresponding to each monitoring period from the transformer environment video, and extract the environmental characteristic parameters corresponding to the transformer from the to-be-recognized images based on a preset extraction model. The extraction formula of the preset extraction model is:

[0168] ;

[0169] Among them, px is the environmental feature parameter corresponding to the xth image to be identified. is the activation function of the preset extraction model, m is the connection index between the xth image to be recognized and the zth layer neuron of the preset extraction model, is the weight parameter of the neurons in the zth layer of the preset extraction model, is the RGB feature value of each pixel in the xth image to be identified, and bz is the bias value of the zth layer neuron of the preset extraction model;

[0170] The transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter are integrated based on each monitoring period to obtain transformer characteristic parameters, and the transformer is operated and maintained in the current operation and maintenance cycle according to each transformer characteristic parameter.

[0171] Furthermore, the step of acquiring multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle includes:

[0172] Receiving text data, image data, audio data and video data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, and determining whether the text data, image data, audio data and video data all have complete data packet numbers;

[0173] If all of them have complete data packet numbers, format conversion is performed on the text data, image data, audio data and video data to form the multimodal data;

[0174] If there is data to be transmitted that does not have a complete data packet number among the text data, image data, audio data and video data, a transmission request is sent to the detection component based on the missing data packet number of the data to be transmitted to obtain a complete data packet to form the multimodal data.

[0175] Further, after the step of sending a transmission request to the detection component based on the data packet number missing from the data to be transmitted to obtain a complete data packet to form the multimodal data; the processor 1001 may call the control program stored in the memory 1005 and perform the following operations:

[0176] Receive the encrypted data packet sent by the detection component based on the transmission request, and calculate the missing data packet number based on a preset decryption algorithm to obtain a decryption key, wherein the preset decryption algorithm is:

[0177] ;

[0178] Where S represents the decryption key, represents the data volume of the nth data packet, An represents the data volume of the complete data packet, x1 and x2 are the missing data packet numbers, and y1 and y2 are the adjacent data packet number pairs corresponding to the missing data packet numbers;

[0179] Decrypting the encrypted data packet based on the decryption key to obtain a decrypted data packet, and merging the decrypted data packet with the data to be transmitted according to the missing data packet number to form a complete data packet;

[0180] The complete data packet is format converted to form the multimodal data.

[0181] The specific implementation of the photovoltaic power station operation and maintenance system based on multimodal data fusion of the present invention is basically the same as the above-mentioned embodiments of the photovoltaic power station operation and maintenance method based on multimodal data fusion, and will not be repeated here.

[0182] The embodiment of the present invention further provides a medium, which is a readable storage medium having a control program stored thereon, and when the control program is executed by a processor, the steps of the photovoltaic power station operation and maintenance method based on multimodal data fusion are implemented as described above.

[0183] The readable storage medium of the present invention may be a computer-readable storage medium, and its specific implementation is substantially the same as the above-mentioned embodiments of the photovoltaic power station operation and maintenance method based on multimodal data fusion, and will not be described in detail here.

[0184] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

Claims

1. A photovoltaic power station operation and maintenance method based on multimodal data fusion, characterized in that: The photovoltaic power station includes a photovoltaic component, a busbar component, an inverter component, and a detection component. The photovoltaic power station operation and maintenance method includes: Acquire multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, wherein the multimodal data includes text data, image data, audio data, and video data that have been converted into a format; wherein: The text data includes: photovoltaic power generation data and photovoltaic temperature difference data of the photovoltaic component, confluence data of the confluence component, inversion data, heat dissipation data and oil level data of the inversion component; The image data includes: a photovoltaic image of the photovoltaic component, a bus image of the bus component, and an inverter image of the inverter component; The audio data is vibration data of the inverter component, and the video data is a voltage transformation environment video of the inverter component; According to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image, the photovoltaic components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle; According to the fusion processing of the confluence data and the confluence image, the confluence components in the photovoltaic power station are operated and maintained in the current operation and maintenance cycle; Performing operation and maintenance of the inverter components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the inverter data, the heat dissipation data, the oil level data, the inverter image, the vibration data, and the transformer environment video; The step of operating and maintaining the photovoltaic components in the photovoltaic power station in the current operation and maintenance cycle according to the photovoltaic power generation data and the photovoltaic temperature difference data, and the fusion processing of the photovoltaic image includes: Performing grayscale processing on the photovoltaic image and forming a grayscale mean corresponding to each photovoltaic panel in the photovoltaic assembly; Forming matrix rows based on the monitoring period of the current operation and maintenance cycle, and forming matrix columns based on each of the photovoltaic panels; Based on the matrix rows and the matrix columns, the photovoltaic power generation data, the photovoltaic temperature difference data and the grayscale mean are respectively formed into a power generation characteristic matrix, a temperature difference characteristic matrix and a grayscale characteristic matrix; The power generation characteristic matrix, the temperature difference characteristic matrix and the grayscale characteristic matrix are fused and calculated to obtain the panel characteristic parameters of each photovoltaic panel in each monitoring period in the current operation and maintenance cycle, wherein the calculation formula is: ； According to the characteristic parameters of each panel, each photovoltaic panel in the photovoltaic assembly is operated and maintained in the current operation and maintenance cycle; Wherein, P(i, t) represents the panel characteristic parameters of the i-th photovoltaic panel in the monitoring period t, Mit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, Nit represents the matrix value of the i-th column and the t-th row in the temperature difference characteristic matrix, Sit represents the matrix value of the i-th column and the t-th row in the power generation characteristic matrix, f1(t±1) is the comparison function based on the monitoring period, and f2(i±1) is the comparison function based on the photovoltaic panel; The step of performing operation and maintenance of the confluence components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the confluence data and the confluence image includes: Generate a confluence parameter of each confluence node in the confluence component according to the photovoltaic power generation data and the confluence data; Based on the preset image processing model, image features of each confluence image are calculated to obtain a spatial region matrix of each confluence image. The calculation formula is: ； Wherein, Rj represents the spatial region matrix of the j-th confluence image, Hj represents the length of the j-th confluence image, Wj represents the width of the j-th confluence image, G2 represents the coordinate conversion function, G1 represents the feature extraction function, K1 represents the number of convolution kernels of the preset image processing model, k2 represents the number of steps of the preset image processing model, k3 represents the number of channels of the preset image processing model, and T* represents the formation of the converted coordinates and the extracted features into a matrix; Based on the similarity between the spatial region matrices, generating element matrices corresponding to the confluence nodes respectively; Finding the reference element matrix corresponding to each confluence node, and fusing each confluence parameter, each element matrix and each reference element matrix to obtain the confluence characteristic parameter of each confluence node; According to each of the confluence characteristic parameters, each confluence node in the confluence component is operated and maintained in the current operation and maintenance cycle.

2. The photovoltaic power station operation and maintenance method according to claim 1, characterized in that: The inverter assembly includes an inverter, a power distribution cabinet and a transformer; The step of operating and maintaining the inverter components in the photovoltaic power station in the current operation and maintenance cycle according to the fusion processing of the inverter data, the heat dissipation data, the oil level data, the inverter image, the vibration data, and the transformer environment video includes: Performing operation and maintenance of the inverter in a current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data; According to the power distribution data in the inverter data, the power distribution and heat dissipation data in the heat dissipation data, and the fusion processing of the power distribution cabinet image in the inverter image, the power distribution cabinet is operated and maintained in the current operation and maintenance cycle; The transformer is operated and maintained in the current operation and maintenance cycle according to the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the fusion processing of the transformer environment video.

3. The photovoltaic power station operation and maintenance method according to claim 2, characterized in that: The step of performing operation and maintenance of the inverter in the current operation and maintenance cycle according to the inverter box data in the inverter data, the inverter heat dissipation data in the heat dissipation data, the inverter box image in the inverter image, and the fusion processing of the vibration data includes: The inverter box data is generated as the inverter efficiency parameters of each monitoring period in the current operation and maintenance cycle; Acquire weather data in the current operation and maintenance cycle, and generate heat dissipation parameters of the inverter box in each monitoring period in the current operation and maintenance cycle according to the weather data and the inverter heat dissipation data; Comparing the inverter box image with a preset inverter reference image to generate inverter image parameters of the inverter box; Based on a preset vibration algorithm, the vibration data is processed to generate the vibration amplitude parameter of the inverter box in the current operation and maintenance cycle. The calculation formula of the preset vibration algorithm is: ； Where H(A, t) represents the vibration amplitude parameter A within the monitoring period t, k represents the kth vibration data, represents the extraction coefficient of the kth vibration data, represents the amplitude extraction function; The inverter efficiency parameter, heat dissipation parameter, inverter image parameter and vibration amplitude parameter are integrated based on each monitoring period to obtain the inverter characteristic parameter, and the inverter is operated and maintained in the current operation and maintenance cycle according to each inverter characteristic parameter.

4. The photovoltaic power station operation and maintenance method according to claim 2, characterized in that: The step of performing operation and maintenance on the transformer in the current operation and maintenance cycle according to the transformer data in the inverter data, the oil level data, the transformer image in the inverter image, and the fusion processing of the transformer environment video includes: The voltage transformation data and the oil level data are respectively generated as voltage transformation efficiency parameters and oil level change parameters of each monitoring period in the current operation and maintenance cycle; Comparing the transformer image with a preset transformer reference image to generate transformer image parameters of the transformer; Extract the to-be-recognized images corresponding to each monitoring period from the transformer environment video, and extract the environmental characteristic parameters corresponding to the transformer from the to-be-recognized images based on a preset extraction model. The extraction formula of the preset extraction model is: ； Among them, px is the environmental feature parameter corresponding to the xth image to be identified. is the activation function of the preset extraction model, m is the connection index between the xth image to be recognized and the zth layer neuron of the preset extraction model, is the weight parameter of the neurons in the zth layer of the preset extraction model, is the RGB feature value of each pixel in the xth image to be identified, and bz is the bias value of the zth layer neuron of the preset extraction model; The transformer efficiency parameter, oil level change parameter, transformer image parameter and environmental characteristic parameter are integrated based on each monitoring period to obtain transformer characteristic parameters, and the transformer is operated and maintained in the current operation and maintenance cycle according to each transformer characteristic parameter.

5. The photovoltaic power station operation and maintenance method according to any one of claims 1 to 4, characterized in that: The step of acquiring multimodal data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle includes: Receiving text data, image data, audio data and video data corresponding to the photovoltaic power station collected by the detection component during the current operation and maintenance cycle, and determining whether the text data, image data, audio data and video data all have complete data packet numbers; If all of them have complete data packet numbers, format conversion is performed on the text data, image data, audio data and video data to form the multimodal data; If there is data to be transmitted that does not have a complete data packet number among the text data, image data, audio data and video data, a transmission request is sent to the detection component based on the missing data packet number of the data to be transmitted to obtain a complete data packet to form the multimodal data.

6. The photovoltaic power station operation and maintenance method according to claim 5, characterized in that: The step of sending a transmission request to the detection component based on the missing data packet number of the data to be transmitted to obtain a complete data packet to form the multimodal data includes: Receive the encrypted data packet sent by the detection component based on the transmission request, and calculate the missing data packet number based on a preset decryption algorithm to obtain a decryption key, wherein the preset decryption algorithm is: ； Where S represents the decryption key, represents the data volume of the nth data packet, An represents the data volume of the complete data packet, x1 and x2 are the missing data packet numbers, and y1 and y2 are the adjacent data packet number pairs corresponding to the missing data packet numbers; Decrypting the encrypted data packet based on the decryption key to obtain a decrypted data packet, and merging the decrypted data packet with the data to be transmitted according to the missing data packet number to form a complete data packet; The complete data packet is format converted to form the multimodal data.

7. A photovoltaic power station operation and maintenance system based on multimodal data fusion, characterized in that: The photovoltaic power station operation and maintenance system based on multimodal data fusion includes: a memory, a processor, a communication bus, and a control program stored in the memory: The communication bus is used to realize the connection and communication between the processor and the memory; The processor is used to execute the control program to implement the steps of the photovoltaic power station operation and maintenance method based on multimodal data fusion as described in any one of claims 1 to 6.

8. A medium, characterized in that The medium is a readable storage medium, and a control program is stored on the readable storage medium. When the control program is executed by the processor, the steps of the photovoltaic power station operation and maintenance method based on multimodal data fusion as described in any one of claims 1 to 6 are implemented.

Images