Integrated energy system content output method, system and device based on large language model

Abstract

The invention relates to the technical field of energy systems, and discloses an integrated energy system content output method, system and device based on a large language model. Firstly, a target page image of a current operation page of the integrated energy system is acquired; secondly, performing vectorization operation on the target page image, and extracting key features from the target page image to obtain a target operation vector; and finally, based on a large language model, according to the private domain knowledge base and the target operation vector, generating and outputting a target content sequence. Through the implementation mode, when the user interacts with the system, operation and problems of the user can be analyzed in real time, and the problems, namely the target content sequence, interested by the user are updated in a personalized mode in real time. The personalized recommendation mechanism not only ensures that the user obtains the latest and most relevant information, but also greatly improves the user experience and satisfaction. When facing complex problems, a user can find a solution more efficiently, so that the operation time and cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of energy systems, and in particular to a method, system and device for outputting content of an integrated energy system based on a large language model. Background Art

[0002] The user interface in the integrated energy system is complex and contains a large amount of information, covering multiple business scenarios such as photovoltaics, electricity, and natural gas. Users often encounter problems when operating in different interfaces and need timely and accurate guidance.

[0003] In the related art, common problems are bound to front-end components through manual maintenance so as to provide recommendations when users encounter related problems. However, the manual maintenance method is costly and relies on predefined rules and relationships, lacking intelligent and personalized recommendation capabilities. Summary of the invention

[0004] The embodiments of this specification are intended to solve at least one of the technical problems in the related art to a certain extent. To this end, the embodiments of this specification propose a method, system and device for outputting content of an integrated energy system based on a large language model.

[0005] The embodiment of this specification provides a method for outputting content of an integrated energy system based on a large language model, the method comprising:

[0006] Acquire a target page image of a current operation page of the integrated energy system;

[0007] By performing a vectorization operation on the target page image, a target operation vector is obtained;

[0008] Based on the large language model, according to the private domain knowledge base and the target operation vector, a target content sequence is generated and output.

[0009] In one embodiment, the generating and outputting a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector includes:

[0010] Based on the large language model, according to the private domain knowledge base and the target operation vector, a target question sequence is generated and output.

[0011] In one embodiment, the step of performing a vectorization operation on the target page image to obtain a target operation vector includes:

[0012] Get the operation position information of the current operation page;

[0013] A vectorization operation is performed on the target page image based on the operation position information to obtain the target operation vector.

[0014] In one embodiment, generating a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector includes:

[0015] Inputting the target operation vector into a large language model, and generating a target vector based on a result obtained by comparing the target operation vector with a private domain knowledge base;

[0016] The large language model generates an extended vector using the target vector;

[0017] The target content sequence is generated by processing the target vector and the extension vector.

[0018] In one embodiment, the private domain knowledge base includes a plurality of preset content vectors, and the generating of the target vector based on the result obtained by comparing the target operation vector with the private domain knowledge base includes:

[0019] Perform similarity calculation based on the target operation vector and the preset content vector to obtain a similar data list;

[0020] A preset content vector corresponding to target similar data satisfying a preset screening condition in the similar data list is determined as the target vector.

[0021] In one embodiment, the method further comprises:

[0022] Get at least one additional page image;

[0023] Obtaining an additional operation vector by performing a vectorization operation on the at least one additional page image;

[0024] The step of generating and outputting a target content sequence based on the private domain knowledge base and the target operation vector based on the large language model includes: inputting the target operation vector and the additional operation vector into the large language model, processing them based on the private domain knowledge base, and generating a target vector and a recommendation vector;

[0025] The target content sequence is generated and output by processing the target vector and the extension vector.

[0026] The embodiment of this specification provides a comprehensive energy system content output system based on a large language model, and the system includes:

[0027] The multimodal perception module is used to obtain a target operation vector by performing vectorization operation on the target page image;

[0028] The user intention perception module is used to generate and output a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector.

[0029] The embodiment of this specification provides a comprehensive energy system content output device based on a large language model, the device comprising:

[0030] A page image acquisition module, used to acquire a target page image of a current operation page of the integrated energy system;

[0031] A vectorization operation module, used for obtaining a target operation vector by performing a vectorization operation on the target page image;

[0032] The content sequence generation output module is used to generate and output a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector.

[0033] An embodiment of the present specification provides a computer device, which includes: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the steps of the method described in any of the above embodiments.

[0034] The embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0035] An embodiment of the present specification provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method described in any one of the above embodiments.

[0036] In the above-mentioned implementation of the specification, first, the target page image of the current operation page of the integrated energy system is obtained. Then, by performing vectorization operation on the target page image, key features are extracted from the target page image to obtain the target operation vector. Finally, based on the large language model, the target content sequence is generated and output according to the private domain knowledge base and the target operation vector. By utilizing the powerful natural language processing capabilities of the large model, the user's operations and problems can be analyzed in real time, and the content can be dynamically generated and updated to ensure that the user obtains the latest and most accurate content information. Through the above-mentioned implementation, when the user interacts with the system, its operations and problems can be analyzed in real time, and the user's questions of interest, i.e., the target content sequence, can be updated in real time and personalized. This personalized recommendation mechanism not only ensures that users obtain the latest and most relevant information, but also greatly improves user experience and satisfaction. When faced with complex problems, users can find solutions more efficiently, thereby reducing the time and cost of operation. In addition, the dependence on manual maintenance is significantly reduced through automation, reducing human resource investment and operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a method for outputting content of an integrated energy system based on a large language model provided for an implementation of this specification;

[0038] Figure 2 A flowchart of a method for outputting content of an integrated energy system based on a large language model provided for an implementation of this specification;

[0039] Figure 3 A schematic diagram of a process for obtaining a target operation vector provided in an embodiment of this specification;

[0040] Figure 4 A schematic diagram of a process for generating a target content sequence provided in an embodiment of this specification;

[0041] Figure 5 A schematic diagram of a process for obtaining a target vector provided in an embodiment of this specification;

[0042] Figure 6 A schematic diagram of a process for generating a target content sequence provided in an embodiment of this specification;

[0043] Figure 7 A schematic diagram of the process of establishing a private domain knowledge base provided in the implementation mode of this specification;

[0044] Figure 8 A schematic diagram of a comprehensive energy system content output device based on a large language model provided for an embodiment of this specification;

[0045] Fig. 9 An internal structural diagram of a computer device provided for an embodiment of this specification. DETAILED DESCRIPTION

[0046] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0047] The user interface in the integrated energy system is complex and contains a large amount of information, covering multiple business scenarios such as photovoltaics, electricity, and natural gas. Users often encounter problems when operating in different interfaces and need timely and accurate guidance.

[0048] In the related art, common problems are bound to front-end components through manual maintenance, so as to provide recommendations when users encounter related problems. However, the manual maintenance method has the following problems:

[0049] Large and complex information volume: The integrated energy system involves multiple business scenarios. Users may encounter different problems in different interfaces, and manual maintenance and solutions are time-consuming and labor-intensive.

[0050] Multi-scenario adaptability problem: The integrated energy system involves multiple business scenarios such as electricity, natural gas, and renewable energy. Each scenario has its own unique business logic and user needs. It is difficult for the recommendation system to adapt to different business scenarios.

[0051] High manual maintenance costs: The relationship between front-end components and common problems needs to be constantly updated. The maintenance cost is high and errors are easily missed. As time goes by, the maintenance cost increases, affecting the recommendation effect.

[0052] Fast iteration speed: The integrated energy system updates its functions quickly, but the recommendation effect is not intelligent enough, mainly relying on predefined rules and lacking personalization capabilities. The system cannot dynamically adjust according to the user's specific operations and needs, resulting in inaccurate recommendations, poor user experience, and inability to meet diverse needs.

[0053] Personalized recommendation problem: Different users on the same interface have different needs, and personalized recommendations still need to be improved to enhance user experience and satisfaction.

[0054] Based on the above analysis, the embodiment of this specification provides a method for outputting content of an integrated energy system based on a large language model. First, a target page image of the current operation page of the integrated energy system is obtained. Then, by performing vectorization operation on the target page image, key features are extracted from the target page image to obtain a target operation vector. Finally, based on the large language model, a target content sequence is generated and output according to the private domain knowledge base and the target operation vector. By utilizing the powerful natural language processing capabilities of the large model, the user's operations and problems can be analyzed in real time, and content can be dynamically generated and updated to ensure that the user obtains the latest and most accurate content information. Through the above implementation, when the user interacts with the system, its operations and problems can be analyzed in real time, and the user's questions, i.e., the target content sequence, can be updated in real time and personalized. This personalized recommendation mechanism not only ensures that the user obtains the latest and most relevant information, but also greatly improves the user experience and satisfaction. When faced with complex problems, users can find solutions more efficiently, thereby reducing the time and cost of operation. In addition, the dependence on manual maintenance is significantly reduced through automation, reducing the investment in human resources and operating costs.

[0055] The embodiments of this specification provide an application scenario of a method for outputting content of an integrated energy system based on a large language model, and propose a development framework combining a large language model (LLM) and a private domain knowledge base. Figure 1, obtain the target page image of the current operation page of the integrated energy system as the input of the convolutional feature extraction module. Input the target page image into the convolutional feature extraction module for vectorization operation to obtain the target operation vector. Input the target operation vector into the large language model, generate the target vector based on the comparison between the target operation vector and the private domain knowledge base, and then the large language model uses the target vector to generate the recommendation vector. The generator performs multimodal processing on the target vector and the recommendation vector to obtain rich text as the target question and a recommended question sequence, thereby realizing personalized common problem recommendation.

[0056] This specification provides a method for outputting content of an integrated energy system based on a large language model. Figure 2 , the integrated energy system content output method based on the large language model can include the following steps:

[0057] S210 . Obtain a target page image of a current operation page of the integrated energy system.

[0058] S220 , obtaining a target operation vector by performing a vectorization operation on the target page image.

[0059] S230: Generate and output a target content sequence based on the large language model according to the private domain knowledge base and the target operation vector.

[0060] Specifically, first, the system needs to capture the operation page image of the current integrated energy system. This can be achieved through screenshots or image capture technology. Make sure that the acquired image is clear and contains all important interface elements for subsequent processing. After acquiring the target page image, the target page image is vectorized through computer vision technology to obtain the target operation vector. Finally, the large language model is used to understand the user's current operation intention based on the private domain knowledge base and the target operation vector, and generate and output the target content sequence that is relevant to the user's current needs and of interest to the user.

[0061] In some embodiments, the target content sequence includes target content directly related to the user's current needs and a personalized extended content sequence. The large language model is used to understand the user's current operation intention based on the private domain knowledge base and the target operation vector, and a target content sequence related to the user's current needs is generated. Based on the private domain knowledge base and the target operation vector, personalized extended content is generated and output to form a recommended content sequence.

[0062] In the above implementation, first, the target page image of the current operation page of the integrated energy system is obtained. Then, by performing vectorization operation on the target page image, key features are extracted from the target page image to obtain the target operation vector. Finally, based on the large language model, the target content sequence is generated and output according to the private domain knowledge base and the target operation vector. By utilizing the powerful natural language processing capabilities of the large model, the user's operations and problems can be analyzed in real time, and the content can be dynamically generated and updated to ensure that the user obtains the latest and most accurate content information. Through the above implementation, when the user interacts with the system, its operations and problems can be analyzed in real time, and the user's questions, i.e., the target content sequence, can be updated in real time and personalized. This personalized recommendation mechanism not only ensures that users obtain the latest and most relevant information, but also greatly improves user experience and satisfaction. When faced with complex problems, users can find solutions more efficiently, thereby reducing the time and cost of operation. In addition, the automation method significantly reduces the dependence on manual maintenance, reduces the investment in human resources and operating costs.

[0063] In some implementations, generating and outputting a target content sequence based on a large language model according to a private domain knowledge base and a target operation vector may include: generating and outputting a target question sequence based on a large language model according to a private domain knowledge base and a target operation vector.

[0064] Specifically, the target content sequence can be a target question sequence. The target question sequence is intended to further clarify the task requirements or promote the realization of the goal. Therefore, based on the large language model, according to the private domain knowledge base and the target operation vector, the target question sequence is generated and output.

[0065] In some embodiments, the target question sequence includes target questions that directly point to the user's needs and a personalized recommended question sequence. Using the large language model, based on the private domain knowledge base and the target operation vector, the user's current operation intention is understood, and a target question directly related to the user's current needs is generated. Based on the private domain knowledge base and the target operation vector, personalized extended content is generated and output to form a recommended question sequence, which encourages users to think about a wider range of related topics, thereby improving the depth of learning and exploration.

[0066] In the above implementation, a target question sequence is generated and output based on a large language model according to a private domain knowledge base and a target operation vector to improve user experience and satisfaction.

[0067] In some embodiments, see Figure 3 , by performing vectorization operation on the target page image, obtaining the target operation vector, which may include the following steps:

[0068] S310: Obtain operation position information of the current operation page.

[0069] S320: Perform a vectorization operation on the target page image based on the operation position information to obtain a target operation vector.

[0070] Specifically, the user's operation on the web page of the integrated energy system cannot be directly understood by the large model, so it is necessary to convert the target page image of the current operation page into a vector feature that can be understood by the large model. Real-time monitoring of the user's input behavior on the current operation page can include multiple interaction methods such as mouse clicks, sliding, and focus changes. Record the user's operation location information, such as the coordinates of each click of the user. Then, use computer vision technology (such as convolutional neural networks) to extract the key visual features in the target page image. Then, combined with the operation location information of the current operation page, these features are vectorized to generate a vector describing the user's operation and obtain the target operation vector. This vector can reflect the specific area that the user pays attention to on the page. Ultimately, the target operation vector will be used to analyze user behavior and may provide data support for subsequent personalized recommendations.

[0071] In some implementations, pan-energy business products such as photovoltaics and energy storage usually have complex structures and need to display the operation and management of enterprises and equipment. In order to better understand user operations, a feature extraction module is designed to convert user intentions into feature vectors that can be understood by the large model.

[0072] The integrated energy system usually consists of a large number of pages to display information such as power station wiring diagrams and power change curves. A single page consists of a limited number of views. Since an overly complex neural network structure is not required, a convolutional neural network (CNN) can be selected to ensure the efficiency of feature extraction while meeting the requirements of feature extraction.

[0073] Inside the convolutional neural network, the target page image passes through the convolution layer and the pooling layer in sequence. The convolution process is:

[0074]

[0075] Where f() is the Relu activation function; T k,j , are the output and input of the convolution process, k and j are the positions of the convolution kernel, ω u,v is the weight of the convolutional layer, b u,v is the convolution layer bias term of this layer, and the convolution kernel size is M×M. Target page image After the convolution operation, the feature matrix T is obtained k,j In order to remove redundant information in the feature matrix and reduce the feature dimension, the feature matrix T k,j Perform pooling operation:

[0076] F=f(T)

[0077] Among them, f() is the pooling function, which represents the pooling operation, and F is the spatial feature matrix obtained by the convolution operation. By performing convolution and pooling operations on the input data layer by layer, the hidden topological structure between the data is extracted. As the number of layers increases, the extracted features become more and more abstract. Finally, the features extracted by each layer of the network converge in the fully connected layer, and the mapping relationship with the label value is established through the activation function. Through the above operations, the convolutional neural network can use the convolution and pooling process to obtain the local spatial features F of the input matrix, and use the fully connected layer to map the distributed features to the sample label space, generating a higher-level feature C for characterizing the target operation vector. Feature C reduces the impact of the feature position on the classification result.

[0078] Assume that the fully connected layer of the convolutional neural network consists of n cnn neurons, then C t =[c1,c2,L,c ncnn ], c i is the state of the ith neuron. Then the temporal latent feature set extracted from the input feature is C c ={C t |t=1,2,K,N c},N c is the sample set size.

[0079] Exemplarily, the target page image of the ubiquitous business system may be a color picture, which may be represented as an n×n×3 vector through feature extraction.

[0080] It should be noted that the number of layers, number of neurons, activation functions, etc. of the model are adjusted according to the characteristics of private domain data. Based on the pre-trained convolutional neural network, fine-tuning is performed in combination with the private domain knowledge base to optimize the accuracy of feature extraction. The specific fine-tuning details are as follows: The adjustment of the convolutional neural network is guided by evaluating the accuracy of the recommended content, such as adjusting the number of layers, number of neurons, activation functions, etc. of the model.

[0081] In the above implementation, the operation position information of the current operation page is obtained, and a vector operation is performed on the target page image based on the operation position information to obtain a target operation vector, so as to better understand the user's intention, thereby achieving personalized recommendations and improving the user experience.

[0082] In some embodiments, see Figure 4 , based on the large language model according to the private domain knowledge base and the target operation vector, generating the target content sequence may include the following steps:

[0083] S410: Input the target operation vector into the large language model, and generate a target vector based on the result obtained by comparing the target operation vector with the private domain knowledge base.

[0084] S420: The large language model generates an extended vector using the target vector.

[0085] S430 : Generate a target content sequence by processing the target vector and the extended vector.

[0086] Specifically, the target operation vector is input into a large language model (LLM). The large language model compares the target operation vector with the private domain knowledge base. During the comparison process, the model identifies the similarities or differences between the target operation vector and the information in the knowledge base to understand the specific needs of the user. Based on the results obtained by the comparison process, a target vector is generated according to preset screening conditions. In some embodiments, the vector that matches the target operation vector in the private domain knowledge base can be directly used as the target vector. In other embodiments, the vector that matches the target operation vector in the private domain knowledge base can be further processed to obtain the target vector. The large language model uses the target vector to generate an extended vector. The extended vector is generated based on the target vector, and sometimes the user's historical behavior is taken into account to optimize the relevance and personalization of the recommendation. Finally, the target vector and the extended vector are subjected to multimodal content generation such as text and pictures to obtain a target content sequence. For example, the large language model can be a multimodal large language model (MLLMs).

[0087] In some embodiments, the main function of the generator is to convert the features output by the large language model (LLM) into content in multiple modalities, including text, pictures, audio, etc. This means that the generator is not limited to generating text information, but can also output images, videos, and other multimedia formats to enhance the user experience. This capability makes up for the shortcomings of traditional LLM in generating multimodal content, allowing users to obtain richer and more intuitive information.

[0088] Exemplarily, the target operation vector is input into the large language model, and based on the comparison between the target operation vector and the private domain knowledge base, the vector obtained according to the comparison result is processed to obtain the target question of interest to the user.

[0089] It should be noted that, based on the pre-trained large language model, fine-tuning is performed in combination with the private domain knowledge base to optimize the recommendation of content sequences in the integrated energy system scenario. The specific fine-tuning details are as follows: The adjustment of the hyperparameters of the large language model is guided by evaluating the accuracy of the recommended content, thereby optimizing the generalization ability of the large language model. In some embodiments, according to the experimental results, the divergence can be set to 0.1 and the kernel sampling parameter (top_p) can be set to 0.9. This combination achieves a good balance between the divergence and accuracy of the model.

[0090] In the above implementation, the target operation vector is input into the large language model, and a target vector is generated based on the result of comparing the target operation vector and the private domain knowledge base. The large language model uses the target vector to generate an extension vector, and the target content sequence is generated by processing the target vector and the extension vector. By utilizing the powerful natural language processing capabilities of the large model, the user's operations and questions can be analyzed in real time, and questions of interest to the user can be dynamically generated and updated, ensuring that the user obtains the latest and most accurate content information.

[0091] In some embodiments, see Figure 5 The private domain knowledge base includes a plurality of preset content vectors. Based on the result obtained by comparing the target operation vector with the private domain knowledge base, generating the target vector may include the following steps:

[0092] S510: Perform similarity calculation based on the target operation vector and the preset content vector to obtain a similar data list.

[0093] S520: Determine a preset content vector corresponding to target similar data that meets a preset screening condition in the similar data list as a target vector.

[0094] Specifically, a similarity measurement method (such as cosine similarity, Euclidean distance) is selected to calculate the similarity between the target operation vector and the preset content vector. For each preset content vector, the selected similarity measurement method is used to calculate the target operation vector to obtain the similarity. According to the calculated similarity, all preset content vectors are sorted to generate a similar data list, for example, they can be arranged from high to low. A set of filtering conditions are pre-set to filter the content in the similar data list. The similar data list is traversed, and the target similar data that meets the conditions is filtered out according to the preset filtering conditions. For the filtered target similar data, the corresponding preset content vector is extracted therefrom. The extracted preset content vector is determined as the target vector for subsequent recommendation or display.

[0095] In some embodiments, the target operation vector is a feature obtained by normalizing using the function of the activation function (the data range is compressed to between 0 and 1) to eliminate the dimensional differences between different features, so that each feature is compared at the same scale. In this case, the dashed line similarity can be used as a method for similarity calculation.

[0096] In the above implementation, similarity calculation is performed based on the target operation vector and the preset content vector to obtain a similar data list, and the preset content vector corresponding to the target similar data that meets the preset screening conditions in the similar data list is determined as the target vector, and the most relevant content is screened out to improve the user experience.

[0097] In some implementations, the target content sequence is multimodal data, and the multimodal data is at least one of text data and image data.

[0098] Specifically, for the integrated energy system, dynamic adjustments are made according to the specific operations and needs of the user, and the target content sequence is displayed in real time. The target content sequence can be multimodal data, including at least one of text data and image data, thereby providing a richer user experience and providing more intelligent and personalized recommendations.

[0099] In the above implementation, the target content sequence is multimodal data, and the multimodal data is at least one of text data and image data, which significantly improves the user experience, enhances the information delivery effect, and provides more personalized and dynamic recommendations.

[0100] In some embodiments, see Figure 6 The method may further include the following steps:

[0101] S610: Acquire at least one additional page image.

[0102] S620: Obtain an additional operation vector by performing a vectorization operation on at least one additional page image.

[0103] Among them, based on the large language model, according to the private domain knowledge base and the target operation vector, the target content sequence is generated and output, including:

[0104] S630: Input the target operation vector and the additional operation vector into the large language model, process them based on the private domain knowledge base, and generate a target vector and an extended vector.

[0105] S640: Generate and output a target content sequence by processing the target vector and the extended vector.

[0106] Specifically, by considering the user's historical behavior, that is, obtaining the page image corresponding to the operation page before the current operation page, at least one additional page image is obtained, thereby optimizing the relevance and personalization of the recommendation. After obtaining the additional page image, an additional operation vector is obtained by performing a vectorization operation on at least one additional page image. Afterwards, the target operation vector and the additional operation vector are input into the large language model, which can understand the relationship between these vectors, and process them based on the private domain knowledge base to generate a target vector and an extended vector. In this way, the model can not only understand the content of the current page, but also make personalized recommendations based on the user's historical preferences. Finally, multi-modal content generation such as text and pictures is performed on the target vector and the extended vector to obtain a target content sequence.

[0107] In the above implementation, at least one additional page image is obtained, and an additional operation vector is obtained by performing a vectorization operation on at least one additional page image. The target operation vector and the additional operation vector are input into the large language model, and the target vector and the extended vector are processed based on the private domain knowledge base to generate the target vector and the extended vector. The target content sequence is generated by processing the target vector and the extended vector. Through the above implementation, when the user interacts with the system, the user's operations and questions can be analyzed in real time, and the questions that the user is interested in, that is, the target content sequence, can be updated in real time and personalized. This personalized recommendation mechanism not only ensures that the user obtains the latest and most relevant information, but also greatly improves the user experience and satisfaction.

[0108] In some embodiments, see Figure 7 , establish a private domain knowledge base through the following methods:

[0109] S710: Perform a vectorization operation on the preset content text to obtain a preset content vector.

[0110] S720: Store the preset content vector to form a private domain knowledge base.

[0111] In some cases, when building a private domain knowledge base, the usual steps include text segmentation, text vectorization, and vector storage. However, in the integrated energy system, the focus is on the questions that users are interested in, and the paragraph attributes of these questions are relatively obvious. Therefore, when building a private domain knowledge base for an integrated energy system based on a large language model, the text segmentation step can be omitted, and text vectorization and vector storage can be performed directly.

[0112] Specifically, in the process of building a private domain knowledge base, it is first necessary to clarify the common content that users of the integrated energy system are concerned about when using it. These contents may include user inquiries on energy efficiency, cost, usage suggestions, troubleshooting, etc. Therefore, these preset contents must first be collected and organized to form a structured preset content text. Next, these preset content texts are processed to generate preset content vectors. This process usually uses natural language processing technology to convert text into high-dimensional vectors through word embedding or other vectorization methods. These preset content vectors can effectively capture the semantic relationship of the text and obtain preset content vectors. The preset content vector can more effectively match the user's query and other multimodal features (such as images). Finally, the generated preset content vectors are stored to form a private domain knowledge base.

[0113] In some implementations, a suitable Embedding model (such as Word2Vec, BERT) is used to convert the preset content text into a vector representation to obtain a preset content vector.

[0114] For example, in the process of constructing a private domain knowledge base, we must first clarify the common problems that users of the integrated energy system are concerned about during use, such as an overview of equipment operation, daily project operation reports, and which inverter saves the most money, to form a preset question text. Perform a vectorization operation on the preset question text to obtain a preset question vector. The preset question vector is stored to form a private domain knowledge base.

[0115] It should be noted that it is crucial to regularly update the private domain knowledge base to ensure the timeliness and accuracy of the content. For example, a comprehensive review can be conducted every quarter to identify and correct outdated information and introduce new information. This review process includes analysis of user feedback, tracking of industry trends, and evaluation of relevant technology and policy changes.

[0116] In the above implementation, a vectorization operation is performed on the preset content text to obtain a preset content vector, and the preset content vector is stored to form a private domain knowledge base, so that the user's query intention can be analyzed through an algorithm later to provide personalized answers.

[0117] The embodiment of this specification provides a comprehensive energy system content output system based on a large language model, and the system includes:

[0118] The multimodal perception module is used to obtain a target operation vector by performing vectorization operations on the target page image.

[0119] The user intention perception module is used to generate and output the target content sequence based on the private domain knowledge base and the target operation vector based on the large language model.

[0120] Specifically, first, the system needs to capture the operation page image of the current integrated energy system. This can be achieved through screenshots or image capture technology. Make sure that the acquired image is clear and contains all important interface elements for subsequent processing. After acquiring the target page image, the multimodal perception module vectorizes the target page image through computer vision technology to obtain the target operation vector. The user intention perception module uses a large language model to generate target content directly related to the user's current needs and personalized extended content based on the private domain knowledge base and the target operation vector to form a target content sequence. After obtaining the target content sequence, the user intention perception module outputs the target content sequence.

[0121] In the above implementation, the multimodal perception module is used to obtain the target operation vector by performing vectorization operations on the target page image, and the user intention perception module is used to generate and output the target content sequence based on the private domain knowledge base and the target operation vector based on the large language model. Utilizing the powerful natural language processing capabilities of the large model, it is possible to analyze the user's operations and problems in real time, and dynamically generate and update content to ensure that the user obtains the latest and most accurate content information. Through the above implementation, it is possible to analyze the user's operations and problems in real time when the user interacts with the system, and update the user's questions, i.e., the target content sequence, in a personalized manner in real time. This personalized recommendation mechanism not only ensures that users obtain the latest and most relevant information, but also greatly improves user experience and satisfaction. When faced with complex problems, users can find solutions more efficiently, thereby reducing the time and cost of operation. In addition, the automation method significantly reduces the reliance on manual maintenance, reduces the investment in human resources and operating costs.

[0122] In some implementations, the multimodal perception module is further used to: obtain operation position information of the current operation page, and perform vectorization operation on the target page image based on the operation position information to obtain a target operation vector.

[0123] Specifically, the multimodal perception module monitors the user's input behavior on the current operation page in real time, which may include mouse clicks, sliding, focus changes and other interaction methods. Record the user's operation location information, such as the coordinates of each click of the user. Then, the multimodal perception module uses computer vision technology (such as convolutional neural networks) to extract key visual features in the target page map. Next, the multimodal perception module combines the operation location information of the current operation page and vectorizes these features to generate a vector describing the user's operation and obtain the target operation vector. This vector can reflect the specific area that the user pays attention to on the page. Ultimately, the target operation vector will be used to analyze user behavior and may provide data support for subsequent personalized recommendations.

[0124] In the above implementation, the operation position information of the current operation page is obtained, and a vector operation is performed on the target page image based on the operation position information to obtain a target operation vector, so as to better understand the user's intention, thereby achieving personalized recommendations and improving the user experience.

[0125] In some embodiments, the user intention perception module is also used to: input the target operation vector into the large language model, generate a target vector based on the result of comparing the target operation vector and the private domain knowledge base, the large language model uses the target vector to generate an extended vector, and generate a target content sequence by processing the target vector and the extended vector.

[0126] Specifically, the user intention perception module inputs the target operation vector into the large language model (LLM). The large language model compares the target operation vector with the private domain knowledge base. During the comparison process, the model will identify the similarities or differences between the target operation vector and the information in the knowledge base to understand the specific needs of the user. The user intention perception module generates a target vector based on the results obtained by the comparison process and according to preset screening conditions. The large language model uses the target vector to generate an extended vector. The extended vector is generated based on the target vector, and sometimes the user's historical behavior is taken into account to optimize the relevance and personalization of the recommendation. Finally, multi-modal content such as text and pictures is generated for the target vector and the recommendation vector to obtain a target content sequence.

[0127] In the above implementation, the target operation vector is input into the large language model, and a target vector is generated based on the result of comparing the target operation vector and the private domain knowledge base. The large language model uses the target vector to generate an extension vector, and the target content sequence is generated by processing the target vector and the extension vector. By utilizing the powerful natural language processing capabilities of the large model, the user's operations and questions can be analyzed in real time, and content can be dynamically generated and updated to ensure that the user obtains the latest and most accurate content information.

[0128] The embodiment of this specification provides a comprehensive energy system content output device 800 based on a large language model, please refer to Figure 8 The integrated energy system content output device 800 based on the large language model includes: a page image acquisition module 810, a vectorization operation module 820, and a content sequence generation and output module 830.

[0129] The page image acquisition module 810 is used to acquire a target page image of a current operation page of the integrated energy system;

[0130] A vectorization operation module 820, configured to obtain a target operation vector by performing a vectorization operation on the target page image;

[0131] The content sequence generation and output module 830 is used to generate and output a target content sequence based on a large language model according to the private domain knowledge base and the target operation vector.

[0132] For a detailed description of the content output device of the integrated energy system based on a large language model, please refer to the description of the content output method of the integrated energy system based on a large language model in the above text, which will not be repeated here.

[0133] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for outputting content of an integrated energy system based on a large language model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0134] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution disclosed in this specification, and does not constitute a limitation on the computer device to which the solution disclosed in this specification is applied. Specifically, the computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method steps in the above embodiments when executing the computer program.

[0136] An embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0137] One embodiment of the present specification provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can perform the steps of the method of any of the above embodiments.

[0138] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

Claims

1. A method for outputting content of an integrated energy system based on a large language model, characterized in that: The method comprises: Acquire a target page image of a current operation page of the integrated energy system; By performing a vectorization operation on the target page image, a target operation vector is obtained; Based on the large language model, according to the private domain knowledge base and the target operation vector, a target content sequence is generated and output.

2. The method according to claim 1, characterized in that The generating and outputting a target content sequence based on the large language model according to the private domain knowledge base and the target operation vector includes: Based on the large language model, according to the private domain knowledge base and the target operation vector, a target question sequence is generated and output.

3. The method according to claim 1, characterized in that The step of performing a vectorization operation on the target page image to obtain a target operation vector includes: Get the operation position information of the current operation page; A vectorization operation is performed on the target page image based on the operation position information to obtain the target operation vector.

4. The method according to any one of claims 1 to 3, characterized in that: Generate a target content sequence based on the private domain knowledge base and the target operation vector based on the large language model, including: Inputting the target operation vector into a large language model, and generating a target vector based on a result obtained by comparing the target operation vector with a private domain knowledge base; The large language model generates an extended vector using the target vector; The target content sequence is generated by processing the target vector and the extension vector.

5. The method according to claim 4, characterized in that The private domain knowledge base includes a plurality of preset content vectors, and the generating of the target vector based on the result obtained by comparing the target operation vector with the private domain knowledge base includes: Perform similarity calculation based on the target operation vector and the preset content vector to obtain a similar data list; A preset content vector corresponding to target similar data satisfying a preset screening condition in the similar data list is determined as the target vector.

6. The method according to claim 1, characterized in that The method further comprises: Get at least one additional page image; Obtaining an additional operation vector by performing a vectorization operation on the at least one additional page image; The step of generating and outputting a target content sequence based on the private domain knowledge base and the target operation vector based on the large language model includes: inputting the target operation vector and the additional operation vector into the large language model, processing them based on the private domain knowledge base, and generating a target vector and an extended vector; The target content sequence is generated and output by processing the target vector and the extension vector.

7. A comprehensive energy system content output system based on a large language model, characterized in that: The system comprises: The multimodal perception module is used to obtain a target operation vector by performing vectorization operation on the target page image; The user intention perception module is used to generate and output a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector.

8. A device for outputting content of an integrated energy system based on a large language model, characterized in that: The device comprises: A page image acquisition module, used to acquire a target page image of a current operation page of the integrated energy system; A vectorization operation module, used for obtaining a target operation vector by performing a vectorization operation on the target page image; The content sequence generation output module is used to generate and output a target content sequence based on a large language model according to a private domain knowledge base and the target operation vector.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images