Photovoltaic operation and maintenance AI assistant system and application method based on deep learning large model

Abstract

The present invention provides a photovoltaic operation and maintenance AI assistant system and application method based on a deep learning large model, which relates to the technical field of photovoltaic power stations, including obtaining text, pictures, voice information and questions input by users, as well as historical operation and maintenance data. Using a multimodal fusion model, these information are integrated to extract features, and are divided into four types of data: training, access, analysis and reporting through semantic classification technology. Furthermore, these classified data are input into a comprehensive data processing model together with historical data. The model integrates functions such as pre-training, database access, big data analysis and report generation to process and generate detailed operation and maintenance reports. Finally, the information is summarized through a natural language processing model to form comprehensive information, which is sent to the user device to guide operation and maintenance decisions. The present invention optimizes the operation and maintenance management process of photovoltaic power stations through intelligent processing.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic power stations, and in particular to a photovoltaic operation and maintenance AI assistant system and an application method based on a deep learning large model. Background Art

[0002] With the rapid development of artificial intelligence technology, photovoltaic operation and maintenance management is gradually shifting towards intelligence and automation. Traditional photovoltaic operation and maintenance relies on manual inspections and empirical judgments, which is not only inefficient, but also difficult to detect and deal with problems in a timely manner, resulting in high operation and maintenance costs and unsatisfactory results. In order to solve this problem, some technical research has begun to try to apply machine learning and data analysis methods to optimize the operation and maintenance process, such as detecting abnormalities of photovoltaic panels through image recognition technology and using historical data for fault prediction. However, most existing technologies are limited to a single data source and processing method, lacking comprehensive analysis and deep learning capabilities for multi-source information, which limits the level of intelligence in operation and maintenance management and the accuracy of fault prediction.

[0003] Therefore, there is an urgent need for an AI assistant system and application method that can more comprehensively understand and analyze operation and maintenance data, improve the accuracy of fault detection and the efficiency of operation and maintenance decision-making. Summary of the invention

[0004] The purpose of the present invention is to provide a photovoltaic operation and maintenance AI assistant system and method based on a deep learning large model to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] In the first aspect, the present application provides a photovoltaic operation and maintenance AI assistant method based on a deep learning large model, including:

[0006] Acquire first information and second information, wherein the first information includes text, pictures, and voice information input by a user and user question information, and the second information includes historical operation and maintenance data information;

[0007] The first information is sent to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, wherein the preset multimodal fusion model is a model for multimodally fusing text features, picture features, and voice features of the user;

[0008] Performing semantic classification on the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information, and report data semantic information;

[0009] Sending all categories of semantic information and second information to a comprehensive data processing model for processing, wherein the comprehensive data processing model includes a pre-training model, a database access model, a big data analysis model, and a big data report automatic generation model, to obtain data information processed by each model in the comprehensive data processing model;

[0010] The data information processed by each model in the data processing comprehensive model and the user question information are sent to the natural language processing model for comprehensive summary to obtain comprehensive information to answer the user's question, and then fed back to the user through a dialog box.

[0011] In the second aspect, the present application also provides a photovoltaic operation and maintenance AI assistant system based on a deep learning large model, characterized in that it includes:

[0012] An acquisition unit, configured to acquire first information and second information, wherein the first information includes text, picture and voice information input by a user and user question information, and the second information includes historical operation and maintenance data information;

[0013] a fusion unit, configured to send the first information to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, wherein the preset multimodal fusion model is a model for multimodally fusing text features, image features, and voice features of a user;

[0014] A classification unit, used for semantically classifying the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information and report data semantic information;

[0015] A processing unit, used to send all categories of semantic information and second information to a data processing comprehensive model for processing, wherein the data processing comprehensive model includes a pre-training model, a database access model, a big data analysis model, and a big data report automatic generation model, to obtain data information processed by each model in the data processing comprehensive model;

[0016] The summarizing unit is used to send the data information and user question information processed by each model in the data processing comprehensive model to the natural language processing model for comprehensive summarization, obtain comprehensive information to answer user questions, and feed back to the user through a dialog box.

[0017] The beneficial effects of the present invention are:

[0018] The present invention processes the text, pictures and voice information input by the user through a multimodal fusion model, combines historical operation and maintenance data, and uses deep learning technology to perform semantic classification and comprehensive data processing. The multimodal fusion and deep learning models it adopts can more comprehensively understand and analyze operation and maintenance data, improve the accuracy of fault detection and the efficiency of operation and maintenance decision-making. In addition, the present invention also includes a natural language processing model that can convert the processed data information into a natural language report to provide users with intuitive operation and maintenance management suggestions. This method of comprehensive utilization of multiple artificial intelligence technologies not only improves the intelligence level of photovoltaic operation and maintenance, but also provides strong support for the efficient and safe operation of photovoltaic power stations.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of a method for applying a photovoltaic operation and maintenance AI assistant based on a deep learning large model described in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of the photovoltaic operation and maintenance AI assistant system based on the deep learning big model described in an embodiment of the present invention.

[0023] In the figure: 701, acquisition unit; 702, fusion unit; 703, classification unit; 704, processing unit; 705, summary unit. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Embodiment 1:

[0026] This embodiment provides an application method of a photovoltaic operation and maintenance AI assistant based on a deep learning large model.

[0027] See also Figure 1 , the figure shows that the method includes step S1, step S2, step S3, step S4 and step S5.

[0028] Step S1, obtaining first information and second information, wherein the first information includes text, pictures, voice information and user problem information input by the user, and the second information includes historical operation and maintenance data information;

[0029] It is understandable that in this step, by integrating the first information and the second information, the system can achieve comprehensive monitoring and analysis of the operating status of the photovoltaic power station. For example, through image recognition technology, the system can automatically identify stains or damage on the photovoltaic panel, and the user's problem information helps the system understand the user's specific needs, thereby providing more accurate services. At the same time, the analysis of historical operation and maintenance data can reveal the operation mode and potential risk points of the power station, and provide data support for preventive maintenance. The first information mainly comes from the user's real-time input, including text, pictures and voice information, which can directly reflect the user's needs and the current operating status of the photovoltaic power station. For example, the user may report a fault by uploading a picture of the photovoltaic panel, or ask the current status of the power station by voice. The second information refers to the historical operation and maintenance data, which is usually stored in a database, including the past operation records, maintenance logs and fault handling records of the power station.

[0030] Step S2: sending the first information to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, wherein the preset multimodal fusion model is a model for multimodally fusing text features, picture features, and voice features of the user;

[0031] It can be understood that in this step, the multimodal fusion model can effectively improve the accuracy and robustness of information processing. For example, in the fault diagnosis of photovoltaic power plants, by fusing image and text information, the model can not only identify the physical damage of photovoltaic panels, but also understand the problem description reported by users, thereby providing more accurate fault analysis and maintenance suggestions. In this step, step S2 includes step S21, step S22, step S23, step S24 and step S25.

[0032] Step S21, respectively performing word segmentation, stop word removal, stem extraction, word form restoration and lowercase conversion processing on the text information in the first information, normalizing, resizing and image enhancement processing on the image information in the first information, and performing noise reduction processing on the voice information in the first information to obtain preprocessed first information, wherein the preprocessed first information includes the preprocessed text information, image information and voice information;

[0033] It can be understood that through these preprocessing steps, the system can more accurately understand and analyze the information input by the user, whether it is text, image or voice, and can be converted into high-quality data that is helpful for operation and maintenance decision-making. Such preprocessing not only improves the availability of data, but also enhances the model's understanding and response capabilities to the operation and maintenance status of photovoltaic power plants, providing a solid foundation for subsequent multimodal fusion and intelligent decision-making. Among them, word segmentation is the process of dividing continuous text into independent units, which is especially important for languages ​​such as Chinese that do not have obvious word boundaries. Removing stop words can reduce the noise of subsequent processing, stem extraction and word type restoration principles help the model understand the basic form of words, and lowercase conversion unifies the format of text data, which helps to improve the consistency of text processing. These processes help to improve the quality of text data and the accuracy of subsequent natural language processing tasks.

[0034] Step S22: sending the preprocessed text information to a preset BERT model for text recognition, and encoding the recognized text to obtain feature vector information of the text;

[0035] It can be understood that the BERT model in this step is a pre-trained language model based on the Transformer architecture. Through two-way training, it can capture the contextual information in the text, thereby providing a more accurate semantic understanding of the text.

[0036] The key to the BERT model is that it can consider each word in the context of the entire sentence, which is different from traditional unidirectional models (such as from left to right or from right to left). During text recognition, the BERT model generates an embedding vector for each word, which is processed by the model's multi-layer Transformer encoder to ultimately form a feature vector that integrates contextual information. These feature vectors contain not only the semantic information of the word itself, but also the relationship information between words, providing rich feature representation for subsequent semantic understanding and information processing.

[0037] The application of the BERT model significantly improves the accuracy and depth of text information processing. In the context of photovoltaic operation and maintenance, this means that the system can more accurately understand the user's questions and instructions, thereby providing more targeted operation and maintenance suggestions and solutions. For example, a user may ask a question about the performance of a photovoltaic panel. Through the analysis of the BERT model, the system can not only understand the keywords of the question, but also combine the contextual information to provide a more comprehensive and accurate answer. This deep semantic understanding capability.

[0038] Step S23, sending the preprocessed image information to a pretrained convolutional neural network model for processing, wherein the pretrained convolutional neural network model is a model trained by recognizing texture features, edge features, and color features of historical image information to obtain feature vector information of the image information;

[0039] It is understood that the image information after preprocessing in this step is fed into a pre-trained convolutional neural network model for feature extraction. This model is trained on historical image data and is able to identify features such as texture, edges, and colors in images. These features are crucial for understanding the content of the image because they can reveal the basic structure and visual patterns of the image.

[0040] Pre-trained CNN models usually contain multiple convolutional layers, each equipped with multiple convolutional kernels that can capture different features. For example, some convolutional kernels may be specialized for detecting lines and edges in an image, while others may be used to recognize specific texture patterns. In this way, CNN is able to extract rich feature representations from images, which can then be used for various image analysis tasks such as image classification, object detection, and image segmentation.

[0041] In the invention, the pre-trained CNN model learns general feature extraction capabilities by training on a large-scale data set. In the context of photovoltaic operation and maintenance, this means that the model can identify damage, dirt or other abnormalities in photovoltaic panels to provide support for operation and maintenance decisions. The pre-trained CNN model significantly improves the efficiency and accuracy of image feature extraction. By utilizing pre-trained models, the required training time and computing resources can be reduced, while also improving the model's adaptability to new tasks. This approach is particularly useful when processing image data from photovoltaic power plants, as it can quickly and accurately identify potential problem areas, thereby improving the response speed and quality of operation and maintenance. In addition, the generalization ability of the pre-trained model also means that it can maintain good performance in different photovoltaic power plants and under different environmental conditions.

[0042] Step S24: Based on the ASR technology, the preprocessed voice information is input and converted to obtain text information corresponding to the preprocessed voice information, and the text information corresponding to the processed voice information is input into a preset BERT model for encoding to obtain feature vector information of the voice information;

[0043] It can be understood that the preprocessing step of this step involves noise reduction, framing, windowing and other operations on the speech signal to improve the clarity and recognizability of the speech. The role of the BERT model in this step is to understand and analyze the converted text information. It captures the contextual information in the text through bidirectional training, thereby generating a feature vector that can represent the semantics of the entire text. These feature vectors contain not only the semantic information of the word itself, but also the relationship information between words, providing rich feature representation for subsequent semantic understanding or information processing. Among them, the combination of ASR technology and BERT model greatly improves the accuracy and depth of speech information processing. In the context of photovoltaic operation and maintenance, this means that the system can more accurately understand the questions or instructions raised by users through voice, thereby providing more targeted operation and maintenance suggestions and solutions. For example, a user may ask about the performance status of a photovoltaic panel through voice. ASR technology can convert voice into text, while the BERT model can understand the intent and content of the text, and the system provides a detailed performance report or maintenance suggestion based on this. The application of this technology not only improves the response speed and quality of operation and maintenance, but also provides great convenience for operation and maintenance personnel.

[0044] Step S25: Based on a preset multi-head attention mechanism, the feature vector information of the text, the feature vector information of the image, and the feature vector information of the voice information are fused to obtain a multimodal fusion feature of the first information.

[0045] It can be understood that the core of the multi-head attention mechanism in this step is that it can process multiple attention heads in parallel, and each head can focus on different aspects of the input data. For example, in text processing, different heads may focus on noun phrases, verb phrases, or other language structures; in image processing, they may focus on features such as color, texture, or shape; and in speech processing, attention heads may capture features such as pitch and rhythm. In this way, the model is able to extract the most relevant information from each modality and integrate it into a unified representation. The multi-head attention mechanism enables the model to have higher flexibility and accuracy when processing multimodal data. It improves the model's ability to capture information through parallel processing and feature fusion, allowing the model to understand data at a higher level, so that it can perform better in tasks such as fault diagnosis and performance analysis.

[0046] Among them, the formula of the multi-head attention mechanism is as follows:

[0047]

[0048] in, is a query converted from a different modality, are keys that switch from different modes and is the vectorization of values ​​converted from different modes, is the dimension of the key vector.

[0049] Step S3, semantically classifying the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information and report data semantic information;

[0050] It can be understood that this step enables the system to quickly respond to user needs and provide relevant data and analysis results. For example, if a user asks about the performance of a photovoltaic panel, the system can provide a detailed performance analysis and possible fault diagnosis by analyzing the data semantic information. If the user needs to understand the overall operation status of the power station, the system can provide a comprehensive operation and maintenance report by reporting the data semantic information. In this step, step S3 includes step S31, step S32, step S33, step S34 and step S35.

[0051] Step S31, constructing a feature sequence according to the multimodal fusion feature, and defining the state space of the feature sequence based on a preset hidden Markov model to obtain implicit state spaces of four semantic categories, wherein the implicit state spaces of the four semantic categories include implicit states of the four semantic categories;

[0052] It can be understood that multimodal fusion features are extracted from text, image and speech information, and these features contain rich information, such as the semantic content of text, the visual pattern of image and the acoustic features of speech. These features are constructed into feature sequences, which provide the basis for the subsequent classification. The hidden Markov model is a statistical model that can describe a Markov process with hidden unknown parameters. In this process, we assume that there is a sequence consisting of hidden states and observations, where the hidden states cannot be observed directly, but can be inferred through the observation sequence. In our scenario, the hidden states represent different semantic categories, such as training data, access data, analysis data, and report data.

[0053] Step S32: constructing a transfer matrix of a hidden Markov model based on the hidden state, calculating the transition probability from one semantic category to another semantic category, and obtaining a transfer matrix of a multimodal fusion feature sequence;

[0054] It can be understood that this step first defines the state space, where "state" refers to different semantic categories. Each state corresponds to an implicit semantic category, and the transition probability between states describes the possibility of transitioning from one semantic category to another. These transition probabilities are obtained by analyzing historical data and user behavior patterns, and they constitute the transition matrix of the hidden Markov model. The construction of the transition matrix is ​​based on the assumption that the next state of the system depends not only on the current state, but may also be affected by the previous states. This Markov property assumes that the transition between states is memoryless, that is, the future transition probability depends only on the current state. By constructing the transition matrix, the present invention can capture the sequential characteristics of user behavior, thereby making more accurate predictions of user needs. This not only improves the accuracy of the system's response to user queries, but also enhances the system's flexibility and adaptability in handling complex user interactions.

[0055] Step S33, smoothing the generation probability of the feature sequence according to the Laplace smoothing technique and the transfer matrix to obtain the generation probability of the smoothed feature sequence;

[0056] It can be understood that the Laplace smoothing in this step is a probabilistic smoothing technique commonly used in natural language processing and information retrieval, which aims to solve the calculation bias caused by the zero probability problem in the probabilistic model. In the hidden Markov model, Laplace smoothing is particularly important because the model relies on accurate estimates of state transition probabilities and observation probabilities. Laplace smoothing ensures that no probability value is zero by adding a small constant (usually 1) to each possible count. As a result, the transition probability and observation probability of each state will be slightly increased, thus avoiding the case of zero probability. This method not only solves the zero probability problem, but also helps to improve the generalization ability of the model because it reduces the risk of overfitting.

[0057] Step S34, performing maximum a posteriori probability calculation on the smoothed observation sequence generation probability according to the forward-backward algorithm of the hidden Markov model to obtain the most likely state path of the multimodal feature sequence;

[0058] It can be understood that in this step, the forward-backward algorithm of the hidden Markov model is used to process the multimodal fusion feature sequence after Laplace smoothing. The forward-backward algorithm is a dynamic programming technique used to calculate the most likely hidden state sequence given the model parameters and the observation sequence, that is, to solve the maximum a posteriori probability problem. Among them, the forward algorithm recursively calculates the forward probability at each moment, that is, the probability that the system reaches a certain state given the prefix of the observation sequence and the model parameters. The backward algorithm starts from the end of the observation sequence and recursively calculates the backward probability, that is, the probability of reaching a certain state from the previous state given the suffix of the observation sequence and the model parameters. By combining the forward probability and the backward probability, we use the Viterbi algorithm to find the maximum a posteriori probability path of the entire observation sequence, which is the most likely state sequence of the multimodal feature sequence. The Viterbi algorithm is a greedy algorithm that uses dynamic programming to find the optimal path, which represents the most likely hidden state sequence corresponding to the observation sequence. Among them, the maximum a posteriori probability calculation formula is as follows:

[0059]

[0060] in, represents the estimated values ​​of the model parameters, represents the model parameters, is the probability of the observed data occurring under the assumption of the model parameters, is the prior probability distribution of the model parameters, is the observation data.

[0061] Step S35: Based on the most likely state path of the multimodal feature sequence and the semantic category corresponding to each state path, a semantic classification result of the multimodal fusion feature sequence is obtained.

[0062] It can be understood that this step can identify the intent of user queries and classify them into semantic categories such as training data, access data, analysis data or report data. Such classification not only improves the accuracy of the system's response to user queries, but also enhances the system's flexibility and adaptability in handling complex user interactions.

[0063] Step S4, sending all categories of semantic information and second information to a comprehensive data processing model for processing, wherein the comprehensive data processing model includes a pre-training model, a database access model, a big data analysis model, and a big data report automatic generation model, to obtain data information processed by each model in the comprehensive data processing model;

[0064] It can be understood that this step can transform multimodal semantic information and historical operation and maintenance data into valuable insights and suggestions. For example, in the operation and maintenance management of photovoltaic power plants, this model can help identify potential performance problems, predict future maintenance needs, and automatically generate detailed operation and maintenance reports. This not only improves the efficiency of operation and maintenance, but also enhances the data-driven ability of decision-making. In this step, step S4 includes step S41, step S42, step S43, and step S44.

[0065] Step S41, sending the training data semantic information and the second information to a pre-training model for processing to obtain solution strategy information corresponding to the training data semantic information;

[0066] It is understandable that the pre-trained model in this step is a BERT or GNN neural network model, which has learned rich language features and patterns through pre-training on large-scale text data. In this step, the pre-trained model receives semantic information of semantically classified training data as input, which may include text data such as operation and maintenance logs, historical fault records, and maintenance operation descriptions of photovoltaic power plants. The model can capture complex relationships and semantic features in the text through its deep network structure. In this step, step S41 includes step S411 and step S412.

[0067] Step S411, preprocessing the training data semantic information and the second information, wherein the preprocessing step includes part-of-speech tagging and converting words into vector representations to obtain preprocessed word vector information;

[0068] It can be understood that part-of-speech tagging in this step is a process of marking each word with its corresponding part of speech (such as noun, verb, adjective, etc.). This step is crucial to understanding the grammatical structure in the text because it provides the grammatical role of the word in the sentence. For example, in the operation and maintenance text of a photovoltaic power station, part-of-speech tagging can help the model identify key information such as operating instructions, equipment names, or status descriptions. Converting words to vector representations usually involves using word embedding techniques, such as Word2Vec or BERT's word embedding layer, to map each word or word fragment to a vector in a high-dimensional space. These vectors capture the semantic information and contextual relationships of words, allowing the model to understand and compare the similarities or differences between different words.

[0069] Step S412: Send the preprocessed word vector information to the pre-training model for training, wherein the pre-training model includes a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer performs self-attention calculation on the input word vector, and captures different semantic information through multiple attention heads. The feedforward neural network layer optimizes the capture of different semantic information by multiple attention heads by using the loss function of multi-task learning, and obtains the solution strategy information corresponding to the semantic information of the training data.

[0070] It can be understood that in this step, through the training of the pre-trained model of this structure, we can obtain more accurate and detailed solution strategy information corresponding to the semantic information of the training data. For example, the model may learn that certain photovoltaic equipment is more likely to fail in certain seasons or weather conditions, thereby generating maintenance recommendations in advance. Among them, the multi-head self-attention layer is a core component of the Transformer architecture, which allows the model to learn information in parallel in different representation subspaces. In this layer, each "head" learns different aspects of the input data. For example, one head may focus on the semantic role of words, while another head may capture the syntactic relationship between words. This mechanism enables the model to understand the text from multiple perspectives and enhances the comprehensive grasp of the text semantics. The feedforward neural network layer further processes the output of the multi-head self-attention layer. This layer usually contains multiple fully connected layers, which integrate and abstract the rich semantic information provided by the self-attention layer to generate more advanced feature representations. In this step, the feedforward neural network optimizes the model through the loss function of multi-task learning. Multi-task learning allows the model to learn multiple related tasks simultaneously, which is particularly useful in PV operation and maintenance, as the model may need to simultaneously predict equipment failures, optimize maintenance plans, and generate operation and maintenance reports.

[0071] Step S42: Send the access data semantic information to the database access model for processing to obtain database access result information;

[0072] It can be understood that this step can quickly respond to the user's data access request and provide timely and accurate information. This not only improves the work efficiency of the operation and maintenance team, but also enhances the data support capability of decision-making. In this step, step S42 includes step S421, step S422 and step S423.

[0073] Step S421: Send the access data semantic information to the input layer of the database access model for processing, wherein natural language processing is performed through the LSTM neural network to obtain a query statement corresponding to the access data semantic information;

[0074] It is understandable that the core advantage of the LSTM network in this step is its ability to learn long-term dependencies in data, which is essential for understanding and processing complex patterns in time series data or natural language text. When processing access data semantic information, LSTM is able to take into account the long-distance relationship between words, so as to generate corresponding query statements more accurately. For example, when a user asks a complex query about the performance of a photovoltaic power plant, LSTM is able to identify the keywords in the query and the relationship between them, and generate an accurate database query statement to retrieve the relevant data. Through the query statements generated by LSTM, the database access model can effectively interact with the database and retrieve the data required by the user. This data can then be used for further analysis or presented directly to the user to support decision making. For example, if a user wants to know the performance data of photovoltaic panels within a specific time period, LSTM can help generate an accurate query statement to retrieve this data from the database.

[0075] Step S422: input the query statement corresponding to the access data semantic information into the parsing layer of the database access model for parsing, wherein the correctness of the query statement is verified based on the probabilistic context-free grammar to obtain a verification result;

[0076] Step S423: If the verification result is that the query statement is a correct query statement, the query statement is executed to obtain database access result information.

[0077] It is understood that in this step, the parsing layer uses probabilistic context-free grammar to verify the correctness of the query statement. Context-free grammar parses sentences by assigning probabilities to grammatical rules. In the context of database queries, context-free grammar can be used to verify whether the query statement conforms to predefined grammatical rules, which describe how to correctly construct SQL queries.

[0078] Step S43: sending the analysis data semantic information and the second information to a big data analysis model for processing to obtain fault analysis information corresponding to the analysis data semantic information;

[0079] It is understandable that the application of big data analysis models can significantly improve the accuracy and efficiency of fault diagnosis. The model can not only identify faults that have occurred, but also predict potential fault risks, thereby achieving predictive maintenance. This helps to reduce unexpected downtime and improve the operating efficiency and reliability of the power station. In this step, step S43 includes step S431, step S432, step S433 and step S434.

[0080] Step S431: Decompose the operation and maintenance data in the second information based on an empirical mode decomposition algorithm to obtain frequency component information of multiple preset time periods in each operation and maintenance data, and use the frequency component information of the multiple preset time periods as time series features;

[0081] It can be understood that in this step, the empirical mode decomposition algorithm is an adaptive signal processing method that can decompose complex, nonlinear and non-stationary time series into a series of intrinsic mode functions, which reflect the characteristics of the data at different time scales. Through the empirical mode decomposition algorithm, we can extract the frequency component information of multiple preset time periods in each operation and maintenance data. The decomposition process of the empirical mode decomposition algorithm includes the following key steps: Identify all local maxima and minima in the data sequence. Use interpolation methods (such as cubic spline interpolation) to construct upper and lower envelopes, which pass through all maximum and minimum points respectively. Calculate the average value of the upper and lower envelopes and subtract this average value from the original data to obtain a candidate intrinsic mode function. Check whether this candidate intrinsic mode function meets the conditions of the intrinsic mode function, that is, in a complete cycle, the number of extreme points and zero crossings is equal or does not differ by more than one. If not, the candidate intrinsic mode function is regarded as a new data sequence. Repeat the above process until all intrinsic mode functions are extracted, and the final data sequence is called residual.

[0082] Step S432: performing a dimensionality reduction operation on the time series features based on a principal component analysis method, wherein the features are sorted according to their variances, and time series features with variances greater than a preset threshold are selected to construct a feature vector matrix;

[0083] It can be understood that this step first calculates the feature covariance matrix through principal component analysis, and then solves the eigenvalues ​​and corresponding eigenvectors of the matrix. The eigenvalue represents the variance in the direction of each eigenvector, while the eigenvector defines the transformed spatial coordinate system. We select eigenvectors whose variance is greater than a preset threshold, and these vectors constitute a new feature space, namely the eigenvector matrix. Each column of this matrix represents a principal component, which is the projection of the original time series features in the new space.

[0084] Step S433: training multiple eigenvectors in the eigenvector matrix simultaneously based on a multi-task Bayesian optimization algorithm, wherein the training of multiple eigenvectors in the eigenvector matrix includes equipment fault prediction training and health score training, and dynamically adjusting model parameters according to preset historical operation and maintenance data information to obtain a trained big data analysis model;

[0085] It can be understood that this step uses the multi-task Bayesian optimization algorithm to train the eigenvector matrix. Bayesian optimization is an optimization method based on Bayesian theory, which guides the search process by constructing a probabilistic model of the objective function. In the multi-task setting, we not only consider the optimization of a single task, but also the interaction between tasks, which helps to improve the overall optimization effect. During the training process, we dynamically adjust the model parameters to adapt to the preset historical operation and maintenance data information. This dynamic adjustment is based on the analysis of historical data and the performance of the current eigenvector, with the aim of finding the optimal model parameters to improve the accuracy of fault prediction and health score. The application of the multi-task Bayesian optimization algorithm enables us to effectively train each eigenvector while considering the relationship between tasks. This method is particularly suitable for the operation and maintenance data analysis of photovoltaic power plants because it can process a large amount of time series data and extract useful information for fault diagnosis and performance evaluation. With this method, we can more accurately predict possible equipment failures, evaluate the health status of equipment, and provide data support for operation and maintenance decisions.

[0086] Step S434: Send the analysis data semantic information and the second information to the trained big data analysis model for fault probability prediction and health score calculation, and evaluate the accuracy of the model prediction through the binary cross entropy loss function to obtain the fault analysis information corresponding to the analysis data semantic information.

[0087] It can be understood that in this step, the semantic information of the analysis data and the second information are input into the trained big data analysis model. This information may include the operating parameters of the equipment, historical fault records, maintenance logs, etc. The big data analysis model is able to process and analyze these complex data sets and identify patterns and trends related to equipment performance and failures. Failure probability prediction is to evaluate the possibility of equipment failure in the future through a model. This usually involves analyzing the real-time monitoring data of the equipment status and combining it with historical fault data to predict future risks. The health score calculation is a quantitative assessment of the current health status of the equipment, which can help the operation and maintenance team understand the overall performance of the equipment and determine the priority of maintenance work. In order to evaluate the accuracy of the model prediction, this step uses the binary cross entropy loss function. Binary cross entropy is a common indicator to measure the difference between the model prediction probability distribution and the actual observation data. It is particularly suitable for binary classification problems. In the operation and maintenance analysis of photovoltaic power stations, the binary cross entropy loss function can help us quantify the performance of the model in distinguishing between normal and fault states. The formula of the binary cross entropy loss function is as follows:

[0088]

[0089] in, represents the binary cross entropy loss function value, is the true label, is the model prediction result, is the number of model samples.

[0090] Step S44: Send the report data semantic information, solution strategy information, database access result information and fault analysis information to the big data report automatic generation model for processing to obtain operation and maintenance report information converted into natural language.

[0091] It is understandable that the application of the big data report automatic generation model significantly improves the efficiency and quality of report generation. It can not only generate reports quickly, but also ensure the accuracy and readability of the report content. This is a huge advantage for the operation and maintenance team of the photovoltaic power station, because they can quickly obtain a summary and analysis of key information, so as to make more timely and accurate decisions. In this step, step S44 includes step S441, step S442 and step S443.

[0092] Step S441: Create a report template based on a preset template engine to obtain template information of the operation and maintenance report;

[0093] Step S442: convert the report data semantic information, solution strategy information, database access result information and fault analysis information into a programming language and fill them into the template information of the operation and maintenance report to obtain the operation and maintenance report information;

[0094] Step S443: Combine the operation and maintenance report information according to a preset time length to obtain operation and maintenance report information of different time periods.

[0095] It is understandable that in this step, the semantic information of the report data, the solution strategy information, the database access result information and the fault analysis information are input into the big data report automatic generation model. This information may include the performance indicators of the photovoltaic power plant, the fault diagnosis results, the maintenance suggestions and the historical data comparative analysis. The model needs to be able to understand and process these complex data and convert them into meaningful report content. The model converts the data and analysis results into text through natural language generation (NLG) technology. This process may involve selecting a suitable language style, organizing information structure and generating descriptive text. For example, the model may use a template to build the basic framework of the report, and then fill in the specific content according to the actual data, such as "In the performance analysis of the photovoltaic power plant in May 2024, we observed that the power generation increased by 5% compared with the previous month, which was mainly due to the maintenance work in the third quarter." In addition, the model may also use advanced text generation techniques, such as the sequence-to-sequence model based on Transformer, to generate more fluent and natural text. These models are able to generate coherent narratives based on the input data information and provide detailed analysis and suggestions.

[0096] Step S5: Send the data information and user question information processed by each model in the data processing comprehensive model to the natural language processing model for comprehensive summary, obtain comprehensive information to answer the user's question, and feed it back to the user through a dialog box.

[0097] It can be understood that this step realizes the automated process from data processing to information transmission through the application of the natural language processing model, provides users with timely and accurate operation and maintenance management suggestions, and thus improves the intelligent level of photovoltaic power station operation and maintenance. In this step, step S5 includes step S51 and step S52.

[0098] Step S51: Send the report data semantic information, solution strategy information, database access result information and fault analysis information to a sequence-to-sequence model for encoding and decoding processing, wherein both the encoder and the decoder adopt a Transformer architecture to obtain third information, wherein the third information includes the report data semantic information, solution strategy information, database access result information and fault analysis information converted into natural language;

[0099] Step S52: Perform grey correlation analysis on the third information and the user problem information, and use the information in the third information that has the highest correlation with the user problem information as operation and maintenance report information.

[0100] It can be understood that in this step, a sequence-to-sequence (Seq2Seq) model is used to process and convert data. The Seq2Seq model is a powerful deep learning architecture that can convert input sequences into output sequences, and is particularly suitable for tasks such as machine translation, text summarization, and speech recognition. In this step, the encoder part of the model receives input data, including report data semantic information, solution strategy information, database access result information, and fault analysis information, and encodes this data into an intermediate representation. Then, the decoder part converts these intermediate representations into operation and maintenance report information in natural language form.

[0101] It can be understood that in this step, both the encoder and the decoder use the Transformer architecture, which is a model based on the self-attention mechanism that can handle long-distance dependency problems in sequence data. Transformer captures the relationship between elements by calculating the attention score of each element in the sequence to all other elements. This mechanism enables the model to better understand and process complex data structures. This step also performs grey correlation analysis on the generated third information (i.e., report data converted into natural language) and user problem information. Grey correlation analysis is a statistical method used to evaluate the degree of correlation between factors. By calculating the correlation between the third information and the user problem information, we can determine which information is most relevant to the user problem. The information with the highest correlation will be selected as the final operation and maintenance report information to ensure that the report content is closely related to the actual needs of the user.

[0102] Furthermore, by combining the Seq2Seq model with grey correlation analysis, we are able to generate accurate, relevant and easy-to-understand operation and maintenance reports. These reports not only provide in-depth analysis of the operation and maintenance data of photovoltaic power plants, but also directly respond to user queries, improving the efficiency and quality of operation and maintenance management. The application of this method makes the operation and maintenance decision-making of photovoltaic power plants more data-driven and intelligent, thereby improving the operation efficiency and reliability of power plants. Embodiment 2:

[0103] like Figure 2 As shown, this embodiment provides a photovoltaic operation and maintenance AI assistant system based on a deep learning large model, see Figure 2 The system includes an acquisition unit 701 , a fusion unit 702 , a classification unit 703 , a processing unit 704 and a summary unit 705 .

[0104] The acquisition unit 701 is used to acquire first information and second information, wherein the first information includes text, picture and voice information input by the user and user question information, and the second information includes historical operation and maintenance data information;

[0105] A fusion unit 702 is used to send the first information to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, where the preset multimodal fusion model is a model for multimodally fusing text features, image features, and voice features of a user;

[0106] A classification unit 703 is used to perform semantic classification on the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information and report data semantic information;

[0107] The processing unit 704 is used to send all categories of semantic information and second information to the data processing comprehensive model for processing, wherein the data processing comprehensive model includes a pre-training model, a database access model, a big data analysis model and a big data report automatic generation model, and obtain the data information processed by each model in the data processing comprehensive model;

[0108] The summarizing unit 705 is used to send the data information and user question information processed by each model in the data processing comprehensive model to the natural language processing model for comprehensive summarization, obtain comprehensive information to answer the user's question, and feed it back to the user through a dialog box.

[0109] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0111] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An application method of a photovoltaic operation and maintenance AI assistant based on a deep learning large model, characterized in that: include: Acquire first information and second information, wherein the first information includes text, pictures, and voice information input by a user and user question information, and the second information includes historical operation and maintenance data information; The first information is sent to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, wherein the preset multimodal fusion model is a model for multimodally fusing text features, picture features, and voice features of the user; Performing semantic classification on the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information, and report data semantic information; Sending all categories of semantic information and second information to a comprehensive data processing model for processing, wherein the comprehensive data processing model includes a pre-training model, a database access model, a big data analysis model, and a big data report automatic generation model, to obtain data information processed by each model in the comprehensive data processing model; Send the data information and user question information processed by each model in the data processing comprehensive model to the natural language processing model for comprehensive summary, obtain comprehensive information to answer the user's question, and feed it back to the user through a dialog box; The multimodal fusion features are semantically classified to obtain semantic information of four categories, including: Constructing a feature sequence according to the multimodal fusion feature, and defining the state space of the feature sequence based on a preset hidden Markov model to obtain implicit state spaces of four semantic categories, wherein the implicit state spaces of the four semantic categories include implicit states of the four semantic categories; Based on the hidden state, a transfer matrix of a hidden Markov model is constructed, and a transfer probability from one semantic category to another semantic category is calculated to obtain a transfer matrix of a multimodal fusion feature sequence; The generation probability of the feature sequence is smoothed according to the Laplace smoothing technique and the transfer matrix to obtain the generation probability of the smoothed feature sequence; Performing maximum a posteriori probability calculation on the smoothed observation sequence generation probability according to the forward-backward algorithm of the hidden Markov model to obtain the most likely state path of the multimodal feature sequence; Based on the most likely state path of the multimodal feature sequence and the semantic category corresponding to each state path, the semantic classification result of the multimodal fusion feature sequence is obtained; Among them, all categories of semantic information and second information are sent to the data processing comprehensive model for processing, including: Sending the training data semantic information and the second information to a pre-training model for processing to obtain solution strategy information corresponding to the training data semantic information; Sending the access data semantic information to a database access model for processing to obtain database access result information; Sending the analysis data semantic information and the second information to a big data analysis model for processing to obtain fault analysis information corresponding to the analysis data semantic information; The report data semantic information, solution strategy information, database access result information and fault analysis information are sent to the big data report automatic generation model for processing to obtain operation and maintenance report information converted into natural language.

2. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that , sending the first information to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, including: Performing word segmentation, stop word removal, stem extraction, word form restoration and lowercase conversion processing on the text information in the first information, normalizing, resizing and image enhancement processing on the image information in the first information, and performing noise reduction processing on the voice information in the first information to obtain preprocessed first information, wherein the preprocessed first information includes the preprocessed text information, image information and voice information; The preprocessed text information is sent to the preset BERT model for text recognition, and the recognized text is encoded to obtain the feature vector information of the text; The pre-processed image information is sent to a pre-trained convolutional neural network model for processing, wherein the pre-trained convolutional neural network model is a model trained by recognizing texture features, edge features, and color features of historical image information to obtain feature vector information of the image information; Based on the ASR technology, the preprocessed voice information is input and converted to obtain text information corresponding to the preprocessed voice information, and the text information corresponding to the processed voice information is input into a preset BERT model for encoding to obtain feature vector information of the voice information; Based on a preset multi-head attention mechanism, the feature vector information of the text, the feature vector information of the image and the feature vector information of the voice information are fused to obtain the multimodal fusion features of the first information.

3. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that , sending the training data semantic information and the second information to the pre-training model for processing, and obtaining solution strategy information corresponding to the training data semantic information, including: Preprocessing the training data semantic information and the second information, wherein the preprocessing step includes part-of-speech tagging and converting words into vector representations to obtain preprocessed word vector information; The preprocessed word vector information is sent to the pre-training model for training, where the pre-training model includes a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer performs self-attention calculation on the input word vector and captures different semantic information through multiple attention heads. The feedforward neural network layer optimizes the capture of different semantic information by multiple attention heads by using the loss function of multi-task learning to obtain the solution strategy information corresponding to the semantic information of the training data.

4. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that , sending the access data semantic information to the database access model for processing, and obtaining database access result information, including: Sending the access data semantic information to the input layer of the database access model for processing, wherein natural language processing is performed through an LSTM neural network to obtain a query statement corresponding to the access data semantic information; Inputting the query statement corresponding to the access data semantic information into the parsing layer of the database access model for parsing, wherein the correctness of the query statement is verified based on the probabilistic context-free grammar to obtain a verification result; If the verification result is that the query statement is a correct query statement, the query statement is executed to obtain database access result information.

5. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that , sending the analysis data semantic information and the second information to the big data analysis model for processing, and obtaining fault analysis information corresponding to the analysis data semantic information, including: Decomposing the operation and maintenance data in the second information based on an empirical mode decomposition algorithm to obtain frequency component information of multiple preset time periods in each operation and maintenance data, and using the frequency component information of the multiple preset time periods as time series features; Performing a dimensionality reduction operation on the time series features based on the principal component analysis method, wherein the features are sorted according to their variances, and time series features with variances greater than a preset threshold are selected to construct a feature vector matrix; Based on the multi-task Bayesian optimization algorithm, multiple eigenvectors in the eigenvector matrix are trained simultaneously, wherein the training of multiple eigenvectors in the eigenvector matrix includes equipment fault prediction training and health score training, and the model parameters are dynamically adjusted according to the preset historical operation and maintenance data information to obtain the trained big data analysis model; The analysis data semantic information and the second information are sent to the trained big data analysis model for fault probability prediction and health score calculation, and the accuracy of the model prediction is evaluated by the binary cross entropy loss function to obtain the fault analysis information corresponding to the analysis data semantic information.

6. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that ,The data information processed by each model in the data processing comprehensive model is sent to the natural language processing model for comprehensive summary, and a comprehensive report information that answers the user's questions is obtained, including: Create a report template based on the preset template engine to obtain the template information of the operation and maintenance report; Convert the report data semantic information, solution strategy information, database access result information and fault analysis information into a programming language and fill them into the template information of the operation and maintenance report to obtain the operation and maintenance report information; The operation and maintenance report information is combined according to a preset time length to obtain operation and maintenance report information of different time periods.

7. The application method of the photovoltaic operation and maintenance AI assistant based on the deep learning large model according to claim 1 is characterized in that ,The data information processed by each model in the data processing comprehensive model and the user question information are sent to the natural language processing model for comprehensive summary, and a comprehensive report information that answers the user's question is obtained, including: Sending the report data semantic information, solution strategy information, database access result information, and fault analysis information to a sequence-to-sequence model for encoding and decoding processing, wherein both the encoder and the decoder adopt a Transformer architecture, to obtain third information, wherein the third information includes the report data semantic information, solution strategy information, database access result information, and fault analysis information converted into natural language; A grey correlation analysis is performed on the third information and the user problem information, and the information in the third information that has the highest correlation with the user problem information is used as the operation and maintenance report information.

8. A photovoltaic operation and maintenance AI assistant system based on a deep learning large model, characterized in that: include: An acquisition unit, configured to acquire first information and second information, wherein the first information includes text, picture and voice information input by a user and user problem information, and the second information includes historical operation and maintenance data information; a fusion unit, configured to send the first information to a preset multimodal fusion model for processing to obtain multimodal fusion features of the first information, wherein the preset multimodal fusion model is a model for multimodally fusing text features, image features, and voice features of a user; A classification unit, used for semantically classifying the multimodal fusion features to obtain four categories of semantic information, wherein the four categories of semantic information include training data semantic information, access data semantic information, analysis data semantic information and report data semantic information; A processing unit, used to send all categories of semantic information and second information to a data processing comprehensive model for processing, wherein the data processing comprehensive model includes a pre-training model, a database access model, a big data analysis model, and a big data report automatic generation model, to obtain data information processed by each model in the data processing comprehensive model; A summarizing unit is used to send the data information processed by each model in the data processing comprehensive model and the user question information to the natural language processing model for comprehensive summarization, obtain comprehensive information to answer the user's question, and feed it back to the user through a dialog box; The multimodal fusion features are semantically classified to obtain semantic information of four categories, including: Constructing a feature sequence according to the multimodal fusion feature, and defining the state space of the feature sequence based on a preset hidden Markov model to obtain implicit state spaces of four semantic categories, wherein the implicit state spaces of the four semantic categories include implicit states of the four semantic categories; Based on the hidden state, a transfer matrix of a hidden Markov model is constructed, and a transfer probability from one semantic category to another semantic category is calculated to obtain a transfer matrix of a multimodal fusion feature sequence; The generation probability of the feature sequence is smoothed according to the Laplace smoothing technique and the transfer matrix to obtain the generation probability of the smoothed feature sequence; Performing maximum a posteriori probability calculation on the smoothed observation sequence generation probability according to the forward-backward algorithm of the hidden Markov model to obtain the most likely state path of the multimodal feature sequence; Based on the most likely state path of the multimodal feature sequence and the semantic category corresponding to each state path, the semantic classification result of the multimodal fusion feature sequence is obtained; Among them, all categories of semantic information and second information are sent to the data processing comprehensive model for processing, including: Sending the training data semantic information and the second information to a pre-training model for processing to obtain solution strategy information corresponding to the training data semantic information; Sending the access data semantic information to a database access model for processing to obtain database access result information; Sending the analysis data semantic information and the second information to a big data analysis model for processing to obtain fault analysis information corresponding to the analysis data semantic information; The report data semantic information, solution strategy information, database access result information and fault analysis information are sent to the big data report automatic generation model for processing to obtain operation and maintenance report information converted into natural language.

Images