Operation and maintenance processing scheme generation method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides an operation and maintenance processing scheme generation method and device, equipment, a storage medium and a product, and the method comprises the steps: receiving to-be-analyzed operation and maintenance data; wherein the to-be-analyzed operation and maintenance data comprises to-be-analyzed images and / or to-be-analyzed description information; then, calling a pre-trained large model to analyze the operation and maintenance data to be analyzed, and generating a structured problem summary; and finally, searching historical operation and maintenance data based on the problem summary, and generating a target operation and maintenance processing scheme. Therefore, according to the embodiment of the invention, the operation and maintenance data to be analyzed is analyzed by adopting the large model, the structured problem summary is generated, the historical operation and maintenance data is retrieved according to the structured problem summary, the target operation and maintenance scheme is generated, manual research and judgment and scheme making are not needed, the processing efficiency of the operation and maintenance problem is improved, and the method is adaptive to a large-scale operation and maintenance scene.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, equipment, storage medium, and product for generating operation and maintenance processing solutions. Background Technology

[0002] In common operation and maintenance management scenarios, the core workflow of the operation and maintenance system is as follows: by using the acquisition modules deployed on each device, the system collects the operation and maintenance data (such as hardware status parameters, resource utilization, etc.) and on-site monitoring images in real time; then, the system performs preliminary analysis on the above-mentioned collected operation and maintenance data and monitoring images based on simple preset rules (such as threshold judgment, feature matching, etc.). When the system detects that the data indicators exceed the preset range or that there are abnormal features in the images, it triggers an early warning mechanism to remind the operation and maintenance personnel to intervene.

[0003] This operation and maintenance management method relies heavily on the experience of operation and maintenance personnel. They need to first retrieve the operation and maintenance data and monitoring images corresponding to the early warning for manual analysis, and then formulate targeted solutions based on their own experience. Furthermore, as the scale of the operation and maintenance system continues to expand and the number of connected devices continues to increase, the amount of operation and maintenance data and monitoring images explodes. This method is extremely inefficient in handling operation and maintenance problems and cannot meet the needs of efficient response in large-scale operation and maintenance scenarios. Summary of the Invention

[0004] This invention provides a method, apparatus, device, storage medium, and product for generating operation and maintenance (O&M) solutions. By using a large model to analyze the O&M data to be analyzed, a structured problem summary is generated. Based on this summary, historical O&M data is retrieved to generate a target O&M solution. This eliminates the need for manual analysis and solution formulation, improving the efficiency of handling O&M problems and adapting to large-scale O&M scenarios.

[0005] In a first aspect, embodiments of the present invention provide a method for generating an operation and maintenance processing scheme, including: Receive operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes images to be analyzed and / or descriptive information to be analyzed; The pre-trained large model is invoked to analyze the operation and maintenance data to be analyzed, and a structured problem summary is generated. Based on the problem summary, historical operation and maintenance data is retrieved to generate the target operation and maintenance solution.

[0006] Secondly, embodiments of the present invention also provide an operation and maintenance processing solution generation device, comprising: The data receiving module is used to receive operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes images to be analyzed and / or descriptive information to be analyzed; The summary generation module is used to call a pre-trained large model to analyze the operation and maintenance data to be analyzed and generate a structured problem summary. The solution generation module is used to retrieve historical operation and maintenance data based on the problem summary and generate a target operation and maintenance solution.

[0007] Thirdly, embodiments of the present invention also provide an operation and maintenance processing scheme generation device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the operation and maintenance processing scheme generation method as described in any of the above embodiments.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the operation and maintenance processing scheme generation method as described in any of the above embodiments.

[0009] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, it implements the operation and maintenance processing scheme generation method as described in any of the above embodiments.

[0010] Compared with existing technologies, the operation and maintenance (O&M) processing solution generation method, apparatus, device, storage medium, and product provided in this embodiment of the invention first receive O&M data to be analyzed; wherein, the O&M data to be analyzed includes images and / or descriptive information to be analyzed; then, a pre-trained large model is invoked to analyze the O&M data to be analyzed, generating a structured problem summary; finally, historical O&M data is retrieved based on the problem summary to generate a target O&M processing solution. Therefore, this embodiment of the invention, by using a large model to analyze the O&M data to be analyzed, generating a structured problem summary, and retrieving historical O&M data accordingly, generates a target O&M solution, eliminating the need for manual judgment and solution formulation, thus improving the efficiency of handling O&M problems and adapting to large-scale O&M scenarios. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a method for generating an operation and maintenance processing solution according to an embodiment of the present invention; Figure 2 This is a flowchart of a method for generating an operation and maintenance processing solution according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a RAG design flow provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of text data processing code provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a text data matching code provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of image data processing code provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an operation and maintenance processing solution generation device provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an operation and maintenance processing solution generation device provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] See Figure 1 An embodiment of the present invention provides a method for generating an operation and maintenance processing scheme, see below. Figure 1 The flowchart of the operation and maintenance processing solution generation method shown includes steps S11 to S13: S11. Receive the operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes the image to be analyzed and / or the descriptive information to be analyzed; S12. Call the pre-trained large model to analyze the operation and maintenance data to be analyzed and generate a structured problem summary; S13. Based on the problem summary, retrieve historical operation and maintenance data and generate a target operation and maintenance solution.

[0014] Specifically, the operation and maintenance (O&M) processing solution generation method transforms raw data (images to be analyzed and / or descriptive information to be analyzed) into searchable structured information, and then automatically matches historical O&M data to generate a target O&M processing solution, addressing the pain points of traditional O&M solutions that rely on manual experience and are inefficient. In step S11, the images to be analyzed refer to visualized data in the O&M scenario (such as on-site photos of server indicator lights flashing red, monitoring screenshots, screenshots of equipment panel error pop-ups, data center temperature and humidity monitoring screens, etc.); the descriptive information to be analyzed is a problem description in natural language (such as "Server A indicator light flashes red continuously, accompanied by abnormal fan noise" or "Database connection timed out, error code XXX" entered by O&M personnel). In step S12, a pre-trained large model is used to understand, refine, and structure the raw data from step S11, outputting a structured problem summary. In step S13, the structured problem summary from step S12 is used as search keywords to query historical O&M data, find at least one most matching historical case, and generate a target O&M processing solution based on the found case to solve the system problem.

[0015] For example, see Figure 2 The flowchart shown illustrates the process architecture for generating operation and maintenance solutions. This process includes the following steps: 1. In application scenarios, when maintenance personnel discover problems on-site, they use the mobile application of the intelligent maintenance system to take pictures of the problem site or device screen, briefly describe the problem, and upload the pictures and descriptions to the system. For example, when a server process crashes, the system generates a core dump file to record the memory state at the moment of the crash, and maintenance personnel can take pictures of the core dump file. The system receives the on-site pictures and descriptions uploaded by maintenance personnel, calls the Visual Language Models (VLM) to process the image and text input, and generates a structured problem summary. This structured problem summary includes information such as the problem type, involved devices, and potential impact range.

[0016] 2. Use Retrieval Augmented Generation (RAG) technology to retrieve historical operational data from the knowledge base, select the top-K solutions, sort these solutions by relevance and success rate, and display them to operations personnel on the mobile application interface. Optionally, after completing the first two steps, continue with steps 3 and 4.

[0017] 3. After selecting the top-K solutions, these solutions undergo timeliness evaluation (e.g., whether the solution is compatible with the current device model, software version, and the latest requirements of the operation and maintenance scenario) and risk assessment (e.g., whether the implementation of the solution may lead to service interruption, data loss, or abnormal operation of associated devices). Solutions that pass the evaluation are then re-ranked and pushed to the front end. After reviewing the solutions recommended by the front end, operations and maintenance personnel select the target solution that best suits the current operation and maintenance problem and initiate the execution command. During the manual operation performed by operations and maintenance personnel, the system records each operation step in real time and simultaneously performs a dynamic risk assessment of the current operation: for low-risk operations (e.g., log viewing, parameter querying, and minor configuration adjustments of non-core services), the system automatically executes the corresponding operation; for high-risk operations (e.g., core device restart, database parameter modification, and large-scale resource scheduling), an intervention mechanism is triggered, such as requiring operations and maintenance personnel to conduct a second confirmation after completing the manual operation, or directly coordinating with relevant developers for review and approval before execution.

[0018] 4. Operations and maintenance personnel record the execution results of each operation step through the operations and maintenance management application: If the problem is resolved after the operation, the problem can be marked as "Resolved" and the corresponding issue ticket closed; if the problem persists, other recommended solutions can be tried, or the problem can be escalated directly; if all recommended solutions fail to effectively resolve the problem, the problem is escalated to the expert team for specialized handling. In addition, the system updates the problem status in real time during the execution of operations and maintenance, and synchronously feeds the execution results of each step back to the knowledge base to drive incremental learning. It is worth noting that newly added practical data (including execution effects (obtained through scoring and analysis of execution results)) can be structured using a visual language model to generate structured data, and then vectorized and stored after vector embedding, which can be used for subsequent incremental learning of the knowledge base.

[0019] Compared with existing technologies, the embodiments of the present invention use a large model to analyze the operation and maintenance data to be analyzed, generate a structured problem summary, retrieve historical operation and maintenance data based on this summary, and generate a target operation and maintenance solution. This eliminates the need for manual judgment and solution formulation, improves the efficiency of handling operation and maintenance problems, and is suitable for large-scale operation and maintenance scenarios.

[0020] In the second implementation, based on steps S11-S13, step S13, which involves retrieving historical maintenance data based on the problem summary and generating a target maintenance solution, includes: Based on the problem summary, a pre-built knowledge base is retrieved to obtain the first round of search results; wherein, the first round of search results are a number of operation and maintenance logs that are most relevant to the problem summary. Based on the first search result and the problem summary, a second search basis is generated; The knowledge base is retrieved based on the secondary query criteria to obtain the second round of search results; wherein, the second round of search results are the maintenance logs that are most relevant to the secondary query criteria. Based on the results of the first and second rounds of retrieval, a target operation and maintenance processing plan is generated for the operation and maintenance data to be analyzed.

[0021] Specifically, this implementation method improves the applicability of the generated target operation and maintenance (O&M) solutions by performing at least two rounds of retrieval on the knowledge base based on the question summary and the retrieved search results. A RAG architecture tailored to O&M scenarios is specifically designed, implemented through the following core technical paths: standardized document preprocessing (e.g., converting O&M documents in various formats such as Word, PDF, and images into structured text), intelligent document segmentation (segmenting long documents into knowledge fragments suitable for retrieval based on semantic integrity), incremental knowledge indexing (supporting real-time updates of O&M documents without rebuilding the index), a multi-layered retrieval architecture (combining keyword and semantic hybrid retrieval strategies), and answer quality verification (ensuring answer reliability through multi-model cross-validation). Through these special designs, an intelligent O&M question-and-answer system with strong real-time knowledge updates (e.g., newly added alarm processing documents can be retrieved immediately), high retrieval accuracy (accurately locating relevant O&M solutions), and timely response can be achieved.

[0022] For example, see Figure 3 The RAG design process shown includes: 1. Constructing a vector database (knowledge base) as the retrieval system; 2. Initial query: Using a Visual Language Model (VLM) to process user text (text to be analyzed) and images to be analyzed, generating initial text, extracting key information from the initial text and organizing it into structured data, and encapsulating the structured data into a preset format (such as JSON, a common structured data exchange format), and then using an embedding model to encode the JSON-formatted structured data into an embedding vector (a structured problem summary); the structured problem summary may include device identification (such as device ID), fault type, time range, impact range, etc.; 3. Retrieving the vector database based on the embedding vector obtained in step 2, finding the most relevant data as the first round of search results; then combining the first round of search results with the problem summary and encoding it into an embedding vector for the second round of knowledge base search, obtaining a new round of search results, and combining the search results of all rounds to generate a target operation and maintenance processing solution for the operation and maintenance data to be analyzed. It is worth noting that the number of search rounds in the knowledge base is not limited to the above two rounds; the number of search rounds can be set according to the actual situation and is not limited here. Under the RAG framework based on operation and maintenance scenarios, the knowledge base update latency was reduced from hours to minutes, the retrieval accuracy was improved by 40%, the answer reliability reached 95%, and the system response time was reduced by 60%.

[0023] In the third implementation, based on the second implementation, before retrieving historical operation and maintenance data based on the problem summary and generating the target operation and maintenance solution in step S13, the following is also included: A search index is created for the operation and maintenance logs in the knowledge base, and the operation and maintenance logs are converted into vector representations.

[0024] For example, see Figure 3 An embedding model is used to convert operation and maintenance logs (including event logs and maintenance records) and product knowledge into vector representations, making them easier for semantic search. Inverted indexes and other techniques are then used to build an efficient search index for this vector representation. The embedding model can be a pre-trained language model, such as the Transformer-based bidirectional encoder representation model BERT, or the robustly optimized BERT pre-training method model RoBERTa.

[0025] In the fourth implementation, based on the third implementation, the step of retrieving a pre-built knowledge base according to the problem summary to obtain the first round of search results includes: Using the problem summary as the initial query basis, based on the search index and the vector representation, keyword matching and semantic similarity calculation are performed between the problem summary and the operation and maintenance log, and the first round of search results are obtained by combining the two calculation results. And / or, The step of retrieving the knowledge base based on the secondary query criteria to obtain the second round of search results includes: Based on the search index and the vector representation, the secondary query is performed based on keyword matching and semantic similarity calculation between the query and the operation and maintenance logs. The results of the two calculations are combined to obtain the second round of search results.

[0026] For example, see Figure 3 Based on the search index, the vector database is searched according to the embedding vector indicating the problem summary. Keyword matching and semantic similarity calculations are performed on this embedding vector and operation logs (such as event logs and maintenance records) to retrieve the top-K most relevant data. The first round of search results is analyzed to extract new key information. Combining the newly extracted information with the problem summary, query expansion techniques (such as synonym expansion and entity recognition) are used to expand the combined data. The expanded data is converted to JSON format, and then the JSON data is converted into vector representations to obtain the corresponding embedding vectors. The vector database is searched again to obtain a new round of search results. This round of search may uncover relevant information missed in the first round. All search results are combined to generate a target operation and maintenance solution.

[0027] In the fifth implementation, based on the second implementation, the step of generating a target operation and maintenance processing plan for the operation and maintenance data to be analyzed based on the results of the first round of retrieval and the results of the second round of retrieval includes: The results of the first and second rounds of retrieval are combined, and duplicates are removed to obtain a comprehensive retrieval result. The comprehensive search results are sorted according to the relevance between each search result and its corresponding query criteria. Based on the ranking results, with highly relevant search results as the core, key information from the comprehensive search results is extracted, and text summarization technology is used to integrate the key information to generate a summary. A target operation and maintenance (O&M) processing plan is generated for the O&M data to be analyzed based on the abstract; wherein the target O&M processing plan includes at least one of the problem analysis results, problem solutions, and historical reference information corresponding to the O&M data to be analyzed.

[0028] Specifically, see Figure 3 In each round of retrieval, the relevance between the query criteria and the corresponding search results is calculated. For example, a pre-trained visual language model is used for relevance scoring (other existing technologies can also be used for relevance scoring). For instance, the specific process for generating a target operation and maintenance solution by combining the results of two rounds of retrieval is as follows: 1. After merging the results of the two rounds of retrieval, duplicates are removed, and then the comprehensive retrieval results are sorted according to the relevance score. 2. Information extraction and context integration: Key information is extracted from the sorted retrieval results, and text summarization technology is used to combine the key information with the search engine results to generate a concise summary of the retrieval results. The extracted and summarized information is integrated with the original input (the operation and maintenance data to be analyzed) into a structured context. Optionally, a visual language model (VLM) can be used for data integration. 3. The integrated context is processed using the visual language model (VLM). Based on the retrieved comprehensive retrieval result history, a target operation and maintenance solution is generated (this solution is output as an answer in text form to the operation and maintenance personnel), including analysis of the problem, possible solutions, and relevant historical reference information. Alternatively, the key information from step 2 and the operation and maintenance data to be analyzed can be input into other existing AI models to generate the target operation and maintenance solution.

[0029] In the sixth embodiment, based on any of the above embodiments, the method further includes: Record user feedback on the target operation and maintenance solution; The algorithm related to the retrieval of historical operation and maintenance data is optimized based on the feedback information.

[0030] For example, after the operations and maintenance personnel perform system maintenance using the target operations and maintenance solution, the relevant processing results (such as user feedback) are stored in the database to optimize the retrieval system, such as adjusting the relevance scoring algorithm and updating the index.

[0031] In the seventh implementation, based on any of the above implementations, the maintenance data to be analyzed includes the image to be analyzed, and the large model is a visual language model; The visual language model was trained in the following way: Obtain historical images related to operation and maintenance and their corresponding text data as training samples; Based on a preset masking strategy, the model is pre-trained using the training samples, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical words and numerical values. The training samples are fused with labeled content to form fine-tuned samples; wherein the fused labeled content includes at least one of the following: guiding dialogues in operation and maintenance scenarios, task prompts, and knowledge reasoning bases. The basic visual language model is then subjected to supervised fine-tuning based on the fine-tuned samples to obtain the final visual language model.

[0032] Specifically, the Visual Language Model (VLM) is obtained through pre-training and supervised fine-tuning. The following is an introduction to VLM pre-training and supervised fine-tuning (SFT): 1. VLM Pretraining During pre-training, a domain-specific learning strategy was introduced to improve the model's understanding of specific concepts and terminology in the operations and maintenance domain. The specific implementation method is as follows: a) Pre-training using a large number of operation and maintenance-related image-text pairs.

[0033] (b) Design a specific masking strategy, focusing on masking key technical terms and numerical values. Key technical terms are predefined related words. Specifically, use [mask] to mask keywords such as "error," "warning," and "exception" in QA dialogues. This forces the model to learn the semantic relationships between words, improving the model's understanding and reasoning ability regarding context.

[0034] Understandably, the specific masking strategy for operations and maintenance focuses on the semantic learning of technical terms such as error codes and alarms. The application of this strategy significantly improves the model's adaptability to operations and maintenance scenarios: the accuracy of understanding operations and maintenance terminology increases by over 35%, error code correlation analysis capability improves by 42%, and alarm classification accuracy reaches over 90%.

[0035] c) Introduce pre-training tasks specific to the operation and maintenance scenario, such as alarm classification and severity estimation.

[0036] Specifically, alarm logs are classified and trained, and customized severity prediction tasks are introduced (such as "Central Processing Unit (CPU) utilization exceeds 90%" as a high-risk alarm, "Disk utilization reaches 85%" as a medium-risk alarm, "Service response delay exceeds 2s" as a low-risk alarm, etc.). Multi-objective learning is used to improve the model's understanding of operation and maintenance scenarios.

[0037] 2. Supervised Fine-tuning (SFT) of VLM During supervised fine-tuning, a multi-task learning framework is employed to simultaneously optimize multiple operation and maintenance-related objectives. Supervised fine-tuning enhances the model's ability to generate accurate and executable operation and maintenance instructions, while also strengthening its inference capabilities in complex operation and maintenance scenarios. The specific implementation method is as follows: a) Use high-quality manually labeled data to fine-tune instructions.

[0038] Specifically, the team's operations and maintenance experts should manually annotate the data, especially for guided multi-turn dialogues, using message logs from actual operations and maintenance scenarios to simulate the entire dialogue. For example: On-duty personnel: Upon receiving an attack alert, they should locate the corresponding security contact person, noting the time, location, and source of the attack; Operations and maintenance experts: Are there still instances of CPU utilization exceeding 90%? This process is used to construct a manually annotated multi-turn dataset for fine-tuning the instructions.

[0039] b) Design multiple auxiliary tasks, such as device identification and problem classification.

[0040] Specifically, using corresponding task prompts (such as [device_identify], [classify_query], etc.) allows the model to truly understand the current requirement and ultimately achieve the goal directly without elaborating further. Optionally, the task prompts may include device identification labels and / or problem classification labels.

[0041] c) Introduce reasoning tasks based on operational knowledge to enhance the model's logical reasoning capabilities.

[0042] Specifically, existing operation and maintenance documents and error logs are utilized, and an embedding model is used to vectorize the corresponding documents and records. These vectors are then used to perform retrieval matching and result rearrangement against user input. The rearranged retrieval results are then input into a Large Language Model (LLM), which summarizes and outputs the results. Simultaneously, structured prompts are designed for the LLM to accurately guide the model to output structured results that meet expectations.

[0043] It is worth noting that VLM supervised fine-tuning is not limited to the multi-task fine-tuning in the specific examples above; it can also be performed on a single task, and this is not a limitation here.

[0044] In the eighth embodiment, based on the previous embodiment, the step of supervising the fine-tuning of the basic visual language model based on the fine-tuning samples to obtain the final visual language model includes: A low-rank matrix is ​​added to the original weight matrix of the basic visual language model to form the target weight matrix; Reduce the precision of the base model weights in the target weight matrix and freeze the base model weights; The low-rank matrix is ​​then subjected to supervised fine-tuning based on the fine-tuning samples to obtain the final visual language model.

[0045] Specifically, supervised fine-tuning of the model is achieved by combining low-rank adaptation (LoRA) and quantized low-rank adaptation (QLoRA) techniques, which significantly improves training efficiency and reduces resource requirements while maintaining model performance.

[0046] Furthermore, the step of adding a low-rank matrix to the original weight matrix of the basic visual language model to form the target weight matrix includes: The target weight matrix is ​​formed by superimposing the product of a fixed low-rank matrix and an adjustable low-rank matrix onto the original weight matrix of the basic visual language model. The step of supervising fine-tuning the low-rank matrix based on the fine-tuning samples to obtain the final visual language model includes: supervising fine-tuning the adjustable low-rank matrix based on the fine-tuning samples to obtain the final visual language model.

[0047] For example, a fusion technique based on low-rank adaptation (LoRA) and quantized low-rank adaptation (QLoRA) can be used to achieve efficient fine-tuning of the visual language model (VLM) while reducing resource requirements. Its core technical characteristics and implementation rules are as follows: 1. Weight matrix update: Replace the original weight matrix W with W1 = W + BA, where B and A are low-rank matrices.

[0048] 2. Parameter update rules: θ1 = θ + Δθ, where Δθ only includes the parameters of matrix A.

[0049] 3. Quantization technique: The basic model weights are quantized to 4-bit precision, while the LoRA parameters are kept to 16-bit precision.

[0050] 4. Training objectives: ; in, It represents the answer predicted by the model. It represents visual input. This represents the instruction / prompt text; 'p' is the generator. The conditional probability.

[0051] 5. Parameter update strategy: Only update the LoRA parameters, keeping the base model weights frozen. LoRA,WL. Among them, the LoRA parameter is a lightweight adjustment parameter added to adapt to the operation and maintenance scenario.

[0052] It is worth noting that the fusion technique of low-rank adaptation (LoRA) and quantized low-rank adaptation (QLoRA) is particularly suitable for the pre-training and supervised fine-tuning stages of visual language models.

[0053] In the ninth embodiment, based on the seventh embodiment, the model is pre-trained according to the training samples based on a preset masking strategy, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical terms and values, including: A visual language model framework with a projector is constructed. The projector is used to map the visual features of the historical images to the language embedding space to obtain the projection prediction value. Based on the degree of difference between the projected predicted values ​​and the text data, optimize the parameters of the projector and the language model; Based on a preset masking strategy, the model is pre-trained using the projected predicted values ​​and the text data. A preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model.

[0054] Specifically, a trainable nonlinear projector is introduced into the visual language model to achieve dynamic mapping from visual features to the language embedding space. Specifically, the semantic alignment fine-tuning of the vertical VLM projector is implemented using a joint optimization strategy. By synchronously adjusting the projector parameters and language model parameters, the optimization objective is to minimize the weighted cross-entropy loss between the predicted token and the target token, thereby achieving accurate matching between visual features and language semantics, where the token is a word element. The relevant mathematical expression for the semantic alignment fine-tuning of the projector is as follows: ; ; in, It is a dynamic weighting factor that is dynamically adjusted based on the importance of the token. It is a projection matrix. The input vector is based on the transformation function. (i.e., vision encoder) on the raw data The transformed output vector Through input vector Obtained through conversion. It is weighted cross-entropy.

[0055] The probability distribution function for the model's prediction of the token is as follows: ; in, This represents the feature processing and prediction function of the language model.

[0056] Furthermore, the projector is fine-tuned using operation and maintenance scenario-related fine-tuning data (such as historical operation and maintenance-related images and their corresponding text data). Because the fine-tuning data uses precise and highly adaptable samples to the operation and maintenance scenario, the model can efficiently learn the specific correspondence between images and text in that scenario, achieving more accurate visual-language semantic alignment. This optimization significantly enhances the model's ability to understand the association between professional images (such as monitoring screenshots and device status diagrams) and technical text (such as alarm descriptions and operation and maintenance instructions) in the operation and maintenance scenario, greatly improving the model's scenario adaptability and semantic understanding accuracy in actual operation and maintenance tasks.

[0057] In the ninth embodiment, based on the seventh embodiment, the step of acquiring historical images related to operation and maintenance and their corresponding text data as training samples includes: Acquire historical images related to operation and maintenance, perform text recognition on the historical images, and generate text data corresponding to the historical images; Based on a predefined log-level dictionary, words in the text data that match the predefined log-level dictionary are converted into a preset format; Identify error codes in the text data and highlight them; Training samples are formed based on the text data that has undergone format conversion and highlighting, and the historical images.

[0058] Specifically, historical images include text images related to operations and maintenance. The generation of training samples involves Optical Character Recognition (OCR), log level recognition and standardization, and error code pattern recognition. The following is a brief introduction to log level recognition and standardization and error code pattern recognition: 1. Log Level Recognition and Standardization: A predefined log level dictionary is provided, including but not limited to common log levels such as "ERROR," "WARNING," "INFO," "DEBUG," and "CRITICAL." During text processing, the system converts words matching the vocabulary in this dictionary into a preset format (such as uppercase) to achieve standardized representation of log levels. 2. Error Code Pattern Recognition: Optionally, regular expression technology is used to identify error codes in specific formats. For example, an error code is defined as a string consisting of three uppercase letters, a hyphen, and four numbers, such as "ABC-1234." The regular expression corresponding to this error code can be represented as: r'[AZ]{3}-\d{4}'.

[0059] For example, see Figure 4 The diagram shown illustrates the text data processing code. The process of generating text data in the training samples is as follows: 1. Use OCR technology to perform text recognition on historical images and generate text data corresponding to the historical images.

[0060] 2. Text segmentation: Segment the text after OCR processing into word sequences.

[0061] 3. Log level processing: Iterate through the word sequence. For each word, if it exists in the predefined log level dictionary, convert it to uppercase.

[0062] 4. Error code highlighting: See [link / reference] Figure 5 The illustrated text data matching code diagram re-traverses the processed word sequence, matching each word using a predefined regular expression pattern. If a match is successful, a specific marker is added before and after the word to highlight the error code in subsequent displays.

[0063] 5. Text Recombination: Recombining the processed word sequence into a complete text string.

[0064] The aforementioned technologies enable the creation of precise image-OCR text pairs (i.e., training samples) for the training and optimization of the Visual Language Model (VLM). Through intelligent OCR text alignment technology, log classification accuracy reaches 93%, error code standardization consistency is improved by 56%, and text structuring processing efficiency is increased by 65%. This process significantly enhances the VLM's perception and response sensitivity to specific errors in operational scenarios, enabling it to accurately capture fault error information and output potential error causes. This provides more detailed problem descriptions and support for subsequent knowledge base retrieval, contributing to improved retrieval accuracy.

[0065] Furthermore, the historical image is an image that has undergone image enhancement.

[0066] It should be noted that existing technologies have not been specifically optimized for the image features of operation and maintenance scenarios, resulting in insufficient OCR recognition accuracy of the Visual Language Model (VLM) in complex real-world scenarios such as rotated images and lighting interference. This deficiency can lead to the model outputting incorrect operation and maintenance strategy suggestions, potentially causing serious consequences such as system downtime and business interruption. To address this, this implementation method specifically designs a data fine-tuning preprocessing workflow: on the one hand, it introduces image enhancement technology to optimize the quality of input data; on the other hand, it combines the image understanding and OCR parsing capabilities of the Visual Language Model (VLM) to achieve targeted optimization of large-scale models for vertical categories in operation and maintenance scenarios, thereby solving the aforementioned technical pain points.

[0067] For example, the following describes relevant techniques for image enhancement in data fine-tuning preprocessing: Objective: To improve the model's ability to understand screen images captured in operational scenarios and enhance OCR performance. method: 1) Image preprocessing: Adaptive histogram equalization (CLAHE) algorithm is applied to enhance image contrast.

[0068] 2) Image data enhancement and noise simulation: a) Random brightness, contrast, and hue adjustments.

[0069] b) Reflection processing: Collect and add real screen reflection occlusion samples; create response samples containing difficult-to-identify reflection situations for instruction fine-tuning.

[0070] c) Add specific noises for the operation and maintenance environment: such as the background noise of the computer room and the indicator lights of the equipment.

[0071] Technical effects: The above processing can effectively improve the robustness of the visual encoder and help the model better adapt to various image changes in the real environment; with the help of real reflective samples and supporting processing strategies, the model's resolution performance in reflective occlusion scenarios can be enhanced; and for response samples in scenarios where reflective elements are difficult to identify, the model's ability to cope with extreme interference and output quality can be improved.

[0072] For example, see Figure 6 The image data processing code shown illustrates the image enhancement process, which can involve sequentially performing noise addition, grayscale processing, CLAHE enhancement, and binarization on the original image. After image enhancement, the image information is converted to the standard dataset format required for training the Visual Language Model (VLM) (adapting to the LLAVA dataset specification), which is done in two steps: 1. Fine-tune the format of the JSON file It is responsible for storing all the details of image processing and contains multiple structured fields: image_info: Records the original filename and processing date; paths: The storage path for storing "original image + images from each processing stage"; ocr_results: The path to the associated OCR recognition result file; processing_params: Records parameters for image enhancement (such as CLAHE configuration); metadata: Stores the metadata of the original image (size, color space, file size, etc.).

[0073] 2. LLAVA dataset format conversion The fine-tuned formatted JSON file was adapted to the LLAVA dataset format commonly used for training Visual Language Models (VLM), thus meeting the input specifications for model training. LLAVA stands for Large Language and Vision Assistant.

[0074] The core fields include: id: A unique identifier for the image; image: The storage path of the original image; image_info: Reuses previously recorded image processing details; Conversations: Simulates the dialogue structure of the LLAVA dataset (adapting to the training logic of the Visual Language Model (VLM)).

[0075] In summary, by addressing noise and interference issues in operation and maintenance scene images through "image enhancement," image processing performance in complex environments is significantly improved: image recognition accuracy in highly reflective scenes is increased by 45%, text extraction performance in low-light environments is optimized by 38%, and overall recognition stability is improved by 50%. Furthermore, "JSON formatting" converts the processed data into the standard sample format for training the Visual Language Model (VLM), ultimately providing high-quality, structured training data for operation and maintenance vertical VLMs, helping the model improve its understanding and parsing capabilities of operation and maintenance scene images.

[0076] Compared with existing technologies, the present invention provides a method for generating operation and maintenance (O&M) solutions. First, it receives O&M data to be analyzed, including images and / or descriptive information. Then, it calls a pre-trained large-scale model to analyze the O&M data, generating a structured problem summary. Finally, it retrieves historical O&M data based on the problem summary to generate a target O&M solution. Therefore, the present invention, by using a large-scale model to analyze the O&M data, generates a structured problem summary, retrieves historical O&M data accordingly, and generates a target O&M solution, eliminating the need for manual judgment and solution formulation. This improves the efficiency of handling O&M problems and is suitable for large-scale O&M scenarios.

[0077] See Figure 7 An embodiment of the present invention also provides an operation and maintenance processing solution generation device, comprising: The data receiving module 21 is used to receive operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes images to be analyzed and / or descriptive information to be analyzed; Summary generation module 22 is used to call a pre-trained large model to analyze the operation and maintenance data to be analyzed and generate a structured problem summary; The solution generation module 23 is used to retrieve historical operation and maintenance data based on the problem summary and generate a target operation and maintenance solution.

[0078] In one embodiment, the scheme generation module 23 specifically includes: The initial query unit is used to retrieve a pre-built knowledge base based on the problem summary to obtain the first round of search results; wherein, the first round of search results are a number of operation and maintenance logs that are most relevant to the problem summary. The query basis generation unit is used to combine the first search result and the question summary to generate a secondary query basis; The secondary query unit is used to retrieve the knowledge base based on the secondary query criteria to obtain a second round of retrieval results; wherein the second round of retrieval results are a number of the historical logs and maintenance records that are most relevant to the secondary query criteria. The scheme generation unit is used to generate a target operation and maintenance processing scheme for the operation and maintenance data to be analyzed based on the results of the first round of retrieval and the results of the second round of retrieval.

[0079] In one implementation, an index building module is also included, for: A search index is created for the operation and maintenance logs in the knowledge base, and the operation and maintenance logs are converted into vector representations.

[0080] In one implementation, the initial query unit is specifically used to include: Using the problem summary as the initial query basis, based on the search index and the vector representation, keyword matching and semantic similarity calculation are performed between the problem summary and the operation and maintenance log, and the first round of search results are obtained by combining the two calculation results. And / or, The step of retrieving the knowledge base based on the secondary query criteria to obtain the second round of search results includes: Based on the search index and the vector representation, the secondary query is performed based on keyword matching and semantic similarity calculation between the query and the operation and maintenance logs. The results of the two calculations are combined to obtain the second round of search results.

[0081] In one embodiment, the scheme generation unit is configured to: The results of the first and second rounds of retrieval are combined, and duplicates are removed to obtain a comprehensive retrieval result. The comprehensive search results are sorted according to the relevance between each search result and its corresponding query criteria. Based on the ranking results, with highly relevant search results as the core, key information from the comprehensive search results is extracted, and text summarization technology is used to integrate the key information to generate a summary. A target operation and maintenance (O&M) processing plan is generated for the O&M data to be analyzed based on the abstract; wherein the target O&M processing plan includes at least one of the problem analysis results, problem solutions, and historical reference information corresponding to the O&M data to be analyzed.

[0082] In one implementation, an optimization module is further included, for: Record user feedback on the target operation and maintenance solution; The algorithm related to the retrieval of historical operation and maintenance data is optimized based on the feedback information.

[0083] In one implementation, the maintenance data to be analyzed includes the image to be analyzed, and the large model is a visual language model, which is trained in the following way: Obtain historical images related to operation and maintenance and their corresponding text data as training samples; Based on a preset masking strategy, the model is pre-trained using the training samples, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical words and numerical values. The training samples are fused with labeled content to form fine-tuned samples; wherein the fused labeled content includes at least one of the following: guiding dialogues in operation and maintenance scenarios, task prompts, and knowledge reasoning bases. The basic visual language model is then subjected to supervised fine-tuning based on the fine-tuned samples to obtain the final visual language model.

[0084] In one implementation, the task prompt includes device identification labels and / or problem classification labels.

[0085] In one implementation, the supervised fine-tuning of the basic visual language model based on the fine-tuning samples to obtain the final visual language model includes: A low-rank matrix is ​​added to the original weight matrix of the basic visual language model to form the target weight matrix; Reduce the precision of the base model weights in the target weight matrix and freeze the base model weights; The low-rank matrix is ​​then subjected to supervised fine-tuning based on the fine-tuning samples to obtain the final visual language model.

[0086] In one implementation, the step of adding a low-rank matrix to the original weight matrix of the basic visual language model to form the target weight matrix includes: The target weight matrix is ​​formed by superimposing the product of a fixed low-rank matrix and an adjustable low-rank matrix onto the original weight matrix of the basic visual language model. The step of supervising fine-tuning the low-rank matrix based on the fine-tuning samples to obtain the final visual language model includes: supervising fine-tuning the adjustable low-rank matrix based on the fine-tuning samples to obtain the final visual language model.

[0087] In one implementation, the model is pre-trained based on the training samples according to a preset masking strategy, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical terms and numerical values, including: A visual language model framework with a projector is constructed. The projector is used to map the visual features of the historical images to the language embedding space to obtain the projection prediction value. Based on the degree of difference between the projected predicted values ​​and the text data, optimize the parameters of the projector and the language model; Based on a preset masking strategy, the model is pre-trained using the projected predicted values ​​and the text data. A preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model.

[0088] In one implementation, acquiring historical images related to operation and maintenance and their corresponding text data as training samples includes: Acquire historical images related to operation and maintenance, perform text recognition on the historical images, and generate text data corresponding to the historical images; Based on a predefined log-level dictionary, words in the text data that match the predefined log-level dictionary are converted into a preset format; Identify error codes in the text data and highlight them; Training samples are formed based on the text data that has undergone format conversion and highlighting, and the historical images.

[0089] In one implementation, the historical image is an image enhanced with image enhancement.

[0090] In one implementation, the error code is matched and identified using a predefined regular expression pattern.

[0091] In one implementation, the problem summary includes device identification, fault type, and scope of impact.

[0092] It is worth noting that the specific working process of the operation and maintenance processing scheme generation device described in the embodiments of the present invention can refer to the working process of the operation and maintenance processing scheme generation method described in any of the above embodiments, and will not be repeated here.

[0093] See Figure 8 This invention also provides an operation and maintenance processing solution generation device, including a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the steps as described in the above-described operation and maintenance processing solution generation method embodiment, for example... Figure 1 The steps S11 to S13 described above; or, when the processor 31 executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0094] For example, the computer program can be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the operation and maintenance processing solution generation device. For example, the computer program can be divided into multiple modules. The specific working process of each module can be referred to the working process of the operation and maintenance processing solution model described in the above embodiments, and will not be repeated here.

[0095] The operation and maintenance processing solution generation device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The operation and maintenance processing solution generation device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the operation and maintenance processing solution generation device may also include input / output devices, network access devices, buses, etc.

[0096] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the operation and maintenance processing solution generation equipment, connecting all parts of the equipment via various interfaces and lines.

[0097] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the operation and maintenance processing solution generation device by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback function), etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0098] If the modules of the operation and maintenance processing solution generation equipment are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0099] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the operation and maintenance processing scheme generation method as described in any of the above embodiments.

[0100] Compared with existing technologies, the operation and maintenance (O&M) processing solution generation method, apparatus, device, storage medium, and product provided in this embodiment of the invention first receive O&M data to be analyzed; wherein, the O&M data to be analyzed includes images and / or descriptive information to be analyzed; then, a pre-trained large model is invoked to analyze the O&M data to be analyzed, generating a structured problem summary; finally, historical O&M data is retrieved based on the problem summary to generate a target O&M processing solution. Therefore, this embodiment of the invention, by using a large model to analyze the O&M data to be analyzed, generating a structured problem summary, and retrieving historical O&M data accordingly, generates a target O&M solution, eliminating the need for manual judgment and solution formulation, thus improving the efficiency of handling O&M problems and adapting to large-scale O&M scenarios.

[0101] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating an operation and maintenance processing solution, characterized in that, include: Receive operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes images to be analyzed and / or descriptive information to be analyzed; The pre-trained large model is invoked to analyze the operation and maintenance data to be analyzed, and a structured problem summary is generated. Based on the problem summary, historical operation and maintenance data is retrieved to generate the target operation and maintenance solution.

2. The method for generating an operation and maintenance processing solution as described in claim 1, characterized in that, The step of retrieving historical operation and maintenance data based on the problem summary and generating a target operation and maintenance solution includes: Based on the problem summary, a pre-built knowledge base is retrieved to obtain the first round of search results; wherein, the first round of search results are a number of operation and maintenance logs that are most relevant to the problem summary. Based on the first search result and the problem summary, a second search basis is generated; The knowledge base is retrieved based on the secondary query criteria to obtain the second round of search results; wherein, the second round of search results are the maintenance logs that are most relevant to the secondary query criteria. Based on the results of the first and second rounds of retrieval, a target operation and maintenance processing plan is generated for the operation and maintenance data to be analyzed.

3. The method for generating an operation and maintenance processing solution as described in claim 2, characterized in that, Before retrieving historical operation and maintenance data based on the problem summary to generate the target operation and maintenance solution, the process also includes: A search index is created for the operation and maintenance logs in the knowledge base, and the operation and maintenance logs are converted into vector representations.

4. The method for generating an operation and maintenance processing solution as described in claim 3, characterized in that, The first round of search results, obtained by retrieving a pre-built knowledge base based on the problem summary, includes: Using the problem summary as the initial query basis, based on the search index and the vector representation, keyword matching and semantic similarity calculation are performed between the problem summary and the operation and maintenance log, and the first round of search results are obtained by combining the two calculation results. And / or, The step of retrieving the knowledge base based on the secondary query criteria to obtain the second round of search results includes: Based on the search index and the vector representation, the secondary query is performed based on keyword matching and semantic similarity calculation between the query and the operation and maintenance logs. The results of the two calculations are combined to obtain the second round of search results.

5. The method for generating an operation and maintenance processing solution as described in claim 2, characterized in that, The step of generating a target operation and maintenance processing plan for the operation and maintenance data to be analyzed based on the results of the first round of retrieval and the results of the second round of retrieval includes: The results of the first and second rounds of retrieval are combined, and duplicates are removed to obtain a comprehensive retrieval result. The comprehensive search results are sorted according to the relevance between each search result and its corresponding query criteria. Based on the ranking results, with highly relevant search results as the core, key information from the comprehensive search results is extracted, and text summarization technology is used to integrate the key information to generate a summary. A target operation and maintenance (O&M) processing plan is generated for the O&M data to be analyzed based on the abstract; wherein the target O&M processing plan includes at least one of the problem analysis results, problem solutions, and historical reference information corresponding to the O&M data to be analyzed.

6. The method for generating an operation and maintenance processing solution as described in claim 1, characterized in that, Also includes: Record user feedback on the target operation and maintenance solution; The algorithm related to the retrieval of historical operation and maintenance data is optimized based on the feedback information.

7. The method for generating an operation and maintenance processing solution as described in any one of claims 1 to 6, characterized in that, The maintenance data to be analyzed includes the image to be analyzed, and the large model is a visual language model; The visual language model was trained in the following way: Obtain historical images related to operation and maintenance and their corresponding text data as training samples; Based on a preset masking strategy, the model is pre-trained using the training samples, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical words and numerical values. The training samples are fused with labeled content to form fine-tuned samples; wherein the fused labeled content includes at least one of the following: guiding dialogues in operation and maintenance scenarios, task prompts, and knowledge reasoning bases. The basic visual language model is then subjected to supervised fine-tuning based on the fine-tuned samples to obtain the final visual language model.

8. The method for generating an operation and maintenance processing solution as described in claim 7, characterized in that, The task prompts include device identification labels and / or problem classification labels.

9. The method for generating an operation and maintenance processing solution as described in claim 7, characterized in that, The process of supervising the fine-tuning of the basic visual language model based on the fine-tuned samples to obtain the final visual language model includes: A low-rank matrix is ​​added to the original weight matrix of the basic visual language model to form the target weight matrix; Reduce the precision of the base model weights in the target weight matrix and freeze the base model weights; The low-rank matrix is ​​then subjected to supervised fine-tuning based on the fine-tuning samples to obtain the final visual language model.

10. The method for generating an operation and maintenance processing solution as described in claim 9, characterized in that, The step of adding a low-rank matrix to the original weight matrix of the basic visual language model to form the target weight matrix includes: The target weight matrix is ​​formed by superimposing the product of a fixed low-rank matrix and an adjustable low-rank matrix onto the original weight matrix of the basic visual language model. The step of supervising fine-tuning the low-rank matrix based on the fine-tuning samples to obtain the final visual language model includes: supervising fine-tuning the adjustable low-rank matrix based on the fine-tuning samples to obtain the final visual language model.

11. The method for generating an operation and maintenance processing solution as described in claim 7, characterized in that, The pre-defined masking strategy is used to pre-train the model based on the training samples, and a pre-defined pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical terms and numerical values, including: A visual language model framework with a projector is constructed. The projector is used to map the visual features of the historical images to the language embedding space to obtain the projection prediction value. Based on the degree of difference between the projected predicted values ​​and the text data, optimize the parameters of the projector and the language model; Based on a preset masking strategy, the model is pre-trained using the projected predicted values ​​and the text data. A preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model.

12. The method for generating an operation and maintenance processing solution as described in claim 7, characterized in that, The acquisition of historical images related to operation and maintenance and their corresponding text data as training samples includes: Acquire historical images related to operation and maintenance, perform text recognition on the historical images, and generate text data corresponding to the historical images; Based on a predefined log-level dictionary, words in the text data that match the predefined log-level dictionary are converted into a preset format; Identify error codes in the text data and highlight them; Training samples are formed based on the text data that has undergone format conversion and highlighting, and the historical images.

13. The method for generating an operation and maintenance processing solution as described in claim 12, characterized in that, The historical images are images that have undergone image enhancement.

14. The method for generating an operation and maintenance processing solution as described in claim 12, characterized in that, The error codes are identified using predefined regular expression patterns.

15. The method for generating an operation and maintenance processing solution as described in claim 1, characterized in that, The problem summary includes the device identification, fault type, and scope of impact.

16. A device for generating operation and maintenance solutions, characterized in that, include: The data receiving module is used to receive operation and maintenance data to be analyzed; wherein, the operation and maintenance data to be analyzed includes images to be analyzed and / or descriptive information to be analyzed; The summary generation module is used to call a pre-trained large model to analyze the operation and maintenance data to be analyzed and generate a structured problem summary; The solution generation module is used to retrieve historical operation and maintenance data based on the problem summary and generate a target operation and maintenance solution.

17. The operation and maintenance processing solution generation device as described in claim 16, characterized in that, The scheme generation module specifically includes: The initial query unit is used to retrieve a pre-built knowledge base based on the problem summary to obtain the first round of search results; wherein, the first round of search results are a number of operation and maintenance logs that are most relevant to the problem summary. The query basis generation unit is used to combine the first search result and the question summary to generate a secondary query basis; The secondary query unit is used to retrieve the knowledge base based on the secondary query criteria to obtain a second round of retrieval results; wherein the second round of retrieval results are a number of the historical logs and maintenance records that are most relevant to the secondary query criteria. The scheme generation unit is used to generate a target operation and maintenance processing scheme for the operation and maintenance data to be analyzed based on the results of the first round of retrieval and the results of the second round of retrieval.

18. The operation and maintenance processing solution generation device as described in claim 17, characterized in that, It also includes an index building module, used for: A search index is created for the operation and maintenance logs in the knowledge base, and the operation and maintenance logs are converted into vector representations.

19. The operation and maintenance processing solution generation device as described in claim 18, characterized in that, The initial query unit is specifically used for, including: Using the problem summary as the initial query basis, based on the search index and the vector representation, keyword matching and semantic similarity calculation are performed between the problem summary and the operation and maintenance log, and the first round of search results are obtained by combining the two calculation results. And / or, The step of retrieving the knowledge base based on the secondary query criteria to obtain the second round of search results includes: Based on the search index and the vector representation, the secondary query is performed based on keyword matching and semantic similarity calculation between the query and the operation and maintenance logs. The results of the two calculations are combined to obtain the second round of search results.

20. The operation and maintenance processing solution generation device as described in claim 17, characterized in that, The scheme generation unit is used for: The results of the first and second rounds of retrieval are combined, and duplicates are removed to obtain a comprehensive retrieval result. The comprehensive search results are sorted according to the relevance between each search result and its corresponding query criteria. Based on the ranking results, with highly relevant search results as the core, key information from the comprehensive search results is extracted, and text summarization technology is used to integrate the key information to generate a summary. A target operation and maintenance (O&M) processing plan is generated for the O&M data to be analyzed based on the abstract; wherein the target O&M processing plan includes at least one of the problem analysis results, problem solutions, and historical reference information corresponding to the O&M data to be analyzed.

21. The operation and maintenance processing solution generation device as described in claim 16, characterized in that, It also includes an optimization module for: Record user feedback on the target operation and maintenance solution; The algorithm related to the retrieval of historical operation and maintenance data is optimized based on the feedback information.

22. The operation and maintenance processing solution generation device as described in claim 16, characterized in that, The operational data to be analyzed includes the images to be analyzed, and the large model is a visual language model, which is trained in the following way: Obtain historical images related to operation and maintenance and their corresponding text data as training samples; Based on a preset masking strategy, the model is pre-trained using the training samples, and a preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical words and numerical values. The training samples are fused with labeled content to form fine-tuned samples; wherein the fused labeled content includes at least one of the following: guiding dialogues in operation and maintenance scenarios, task prompts, and knowledge reasoning bases. The basic visual language model is then subjected to supervised fine-tuning based on the fine-tuned samples to obtain the final visual language model.

23. The operation and maintenance processing solution generation device as described in claim 22, characterized in that, The pre-defined masking strategy is used to pre-train the model based on the training samples, and a pre-defined pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model; wherein, the masking strategy focuses on masking key technical terms and numerical values, including: A visual language model framework with a projector is constructed. The projector is used to map the visual features of the historical images to the language embedding space to obtain the projection prediction value. Based on the degree of difference between the projected predicted values ​​and the text data, optimize the parameters of the projector and the language model; Based on a preset masking strategy, the model is pre-trained using the projected predicted values ​​and the text data. A preset pre-training task under the operation and maintenance scenario is introduced during the model pre-training process to train a basic visual language model.

24. The operation and maintenance processing solution generation device as described in claim 22, characterized in that, The acquisition of historical images related to operation and maintenance and their corresponding text data as training samples includes: Acquire historical images related to operation and maintenance, perform text recognition on the historical images, and generate text data corresponding to the historical images; Based on a predefined log-level dictionary, words in the text data that match the predefined log-level dictionary are converted into a preset format; Identify error codes in the text data and highlight them; Training samples are formed based on the text data that has undergone format conversion and highlighting, and the historical images.

25. A device for generating operation and maintenance solutions, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the operation and maintenance processing scheme generation method as described in any one of claims 1 to 15.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the operation and maintenance processing scheme generation method as described in any one of claims 1 to 15.

27. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the operation and maintenance processing scheme generation method as described in any one of claims 1 to 15.

Images