Substation inspection method, device, equipment and medium

Abstract

The invention provides a substation inspection method, device and equipment and a medium. The method comprises the following steps: receiving a natural language instruction which is input by a user and comprises an inspection object of an inspection task and an inspection demand; a large language model is adopted to analyze the natural language instruction, a sub-task sequence corresponding to the inspection task is generated, and the sub-task sequence comprises a first query sub-task used for querying equipment information of inspection equipment for inspecting the inspection object, and a second query sub-task used for querying equipment information of inspection equipment for inspecting the inspection object; the second query subtask is used for querying and executing the model information of the identification model of the inspection requirement, and the execution subtask is used for executing inspection processing; executing each subtask in the subtask sequence to obtain an image including the inspection object through the inspection equipment, and identifying the image through the identification model to obtain an inspection result corresponding to the inspection task; and outputting an inspection result. The method is used for achieving the effects of timely responding to the temporary and specific inspection requirements of the user and improving the intelligent level of the inspection of the transformer substation.

Description

Technical Field

[0001] This application relates to the field of power systems, and in particular to a method, apparatus, equipment and medium for substation inspection. Background Technology

[0002] As the core hub of power system for power transformation, distribution, and transmission, the safe and stable operation of substations is directly related to the reliability of power supply and the normal operation of society and the economy.

[0003] Currently, substation inspections are typically conducted according to a preset inspection frequency and a user-defined inspection strategy. However, while this method can meet the needs of routine inspections, it suffers from a lack of flexibility when faced with ad-hoc inspection tasks (such as the need to inspect electrical equipment in a specific area on short notice).

[0004] Therefore, there is an urgent need for a more intelligent substation inspection method. Summary of the Invention

[0005] This application provides a substation inspection method, device, equipment, and medium to achieve timely response to users' temporary and specific inspection needs and improve the intelligence level of substation inspection.

[0006] In a first aspect, embodiments of this application provide a substation inspection method, including:

[0007] Receive natural language instructions input by the user, the natural language instructions including the inspection objects and inspection requirements of the inspection task;

[0008] The natural language instructions are parsed using a large language model to generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for the inspection object; a second query sub-task for querying the model information of the recognition model for executing the inspection requirement; and an execution sub-task for performing the inspection processing.

[0009] Each subtask in the subtask sequence is executed to acquire an image including the inspection object through the inspection device, and the image is identified through the recognition model to obtain the inspection result corresponding to the inspection task.

[0010] Output the inspection results.

[0011] In one possible implementation, the execution subtasks include a device control task, a model invocation task, and a result analysis task;

[0012] The execution of each subtask in the subtask sequence to acquire an image including the inspected object through the inspection device, and to identify the image through the recognition model to obtain the inspection result corresponding to the inspection task includes:

[0013] Execute the first query subtask to obtain the equipment information of the inspection equipment, which includes the control interface information of the inspection equipment and preset control parameters.

[0014] Execute the second query subtask to obtain the model information of the recognition model, the model information including the calling interface information of the recognition model;

[0015] Based on the control interface information and the control parameters, the control device task is executed to control the inspection device to perform image acquisition, thereby obtaining an image including the inspection object;

[0016] Based on the interface call information, the model call task is executed to control the recognition model to recognize the image and obtain the recognition result;

[0017] The result analysis task is performed to analyze the identification results and obtain the inspection results corresponding to the inspection task.

[0018] In one possible implementation, executing the first query subtask to obtain the equipment information of the inspection equipment for inspecting the inspection object includes:

[0019] A large language model is used to generate the first query conditions based on the inspection object;

[0020] Based on the first query condition, retrieve the equipment information of the inspection equipment for the inspection object from the knowledge base;

[0021] The knowledge base includes the equipment type, unique identifier, and physical location of each power device in the substation, as well as the inspection range, control interface information, and preset control parameters of each inspection device.

[0022] In one possible implementation, executing the second query subtask to obtain model information of the identification model for executing the inspection requirement includes:

[0023] The second query condition is generated based on the inspection requirements using the large language model.

[0024] Based on the second query condition, retrieve the model information of the recognition model that performs the inspection requirement from the pre-established image recognition model management table;

[0025] The image recognition model management table includes the function of each recognition model and the calling interface information.

[0026] In one possible implementation, the knowledge base also includes result interpretation rules corresponding to different requirements of inspection of different power equipment in the substation;

[0027] The subtask sequence also includes a third query subtask for querying and interpreting the result interpretation rules of the recognition model;

[0028] Accordingly, the method further includes:

[0029] The third query subtask is executed to generate third query conditions based on the inspection object and the inspection requirements using the large language model.

[0030] Based on the third query condition, query the knowledge base for result interpretation rules that explain the recognition results of the recognition model;

[0031] Accordingly, the execution of the result analysis task analyzes the identification results to obtain the inspection results corresponding to the inspection task, including:

[0032] The recognition results of the recognition model are interpreted using the large language model based on the result interpretation rules to obtain the interpretation results;

[0033] Based on the interpretation results, the inspection results corresponding to the inspection task are obtained.

[0034] In one possible implementation, obtaining the inspection result corresponding to the inspection task based on the interpretation result includes:

[0035] An analysis report is generated using the large language model based on the natural language instructions, the subtask sequence, the image, the recognition result, and the interpretation result.

[0036] The analysis report shall be used as the inspection result corresponding to the inspection task.

[0037] In one possible implementation, the method further includes:

[0038] When performing control device tasks, the image is subjected to quality detection. If the quality of the image does not meet the preset requirements, the large language model is used to supplement and generate fine-tuning instructions.

[0039] The control parameters are adjusted according to the fine-tuning instructions.

[0040] Secondly, embodiments of this application provide a substation inspection device, comprising:

[0041] The receiving unit is used to receive natural language instructions input by the user, the natural language instructions including the inspection objects and inspection requirements of the inspection task.

[0042] The generation unit is used to parse the natural language instructions using a large language model and generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for inspecting the inspection object; a second query sub-task for querying the model information of the recognition model for executing the inspection requirement; and an execution sub-task for executing the inspection process.

[0043] An execution unit is used to execute each subtask in the subtask sequence to acquire an image including the inspection object through the inspection device, and to identify the image through the recognition model to obtain the inspection result corresponding to the inspection task.

[0044] The output unit is used to output the inspection results.

[0045] In one possible implementation, the execution subtasks include a device control task, a model invocation task, and a result analysis task; the execution unit includes:

[0046] The first query module is used to execute the first query subtask to obtain the equipment information of the inspection equipment. The equipment information includes the control interface information of the inspection equipment and preset control parameters.

[0047] The second query module is used to execute the second query subtask to obtain the model information of the recognition model, wherein the model information includes the calling interface information of the recognition model;

[0048] The first execution module is used to execute the control device task based on the control interface information and the control parameters to control the inspection device to perform image acquisition and obtain an image including the inspection object;

[0049] The second execution module is used to execute the model calling task based on the calling interface information to control the recognition model to recognize the image and obtain the recognition result;

[0050] The third execution module is used to perform the result analysis task to analyze the identification results and obtain the inspection results corresponding to the inspection task.

[0051] In one possible implementation, the first query module is configured to:

[0052] A large language model is used to generate the first query conditions based on the inspection object;

[0053] Based on the first query condition, retrieve the equipment information of the inspection equipment for the inspection object from the knowledge base;

[0054] The knowledge base includes the equipment type, unique identifier, and physical location of each power device in the substation, as well as the inspection range, control interface information, and preset control parameters of each inspection device.

[0055] In one possible implementation, the second query module is used for:

[0056] The second query condition is generated based on the inspection requirements using the large language model.

[0057] Based on the second query condition, retrieve the model information of the recognition model that performs the inspection requirement from the pre-established image recognition model management table;

[0058] The image recognition model management table includes the function of each recognition model and the calling interface information.

[0059] In one possible implementation, the knowledge base also includes result interpretation rules corresponding to different requirements of inspection of different power equipment in the substation; the subtask sequence also includes a third query subtask for querying the result interpretation rules for interpreting the recognition results of the recognition model.

[0060] Accordingly, the execution unit further includes:

[0061] The third query module is used to execute the third query subtask to generate third query conditions based on the inspection object and the inspection requirements using the large language model.

[0062] Based on the third query condition, query the knowledge base for result interpretation rules that explain the recognition results of the recognition model;

[0063] Accordingly, the third execution module specifically includes:

[0064] An explanation module is used to interpret the recognition results of the recognition model based on the result explanation rules using the large language model, and obtain an explanation result;

[0065] The acquisition module is used to acquire the inspection result corresponding to the inspection task based on the interpretation result.

[0066] In one possible implementation, the acquisition module is configured to:

[0067] An analysis report is generated using the large language model based on the natural language instructions, the subtask sequence, the image, the recognition result, and the interpretation result.

[0068] The analysis report shall be used as the inspection result corresponding to the inspection task.

[0069] In one possible implementation, the substation inspection device further includes:

[0070] The detection unit is used to perform quality detection on the image when performing control device tasks. If the quality of the image does not meet the preset requirements, it uses the large language model to supplement and generate fine-tuning instructions; and adjusts the control parameters according to the fine-tuning instructions.

[0071] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor;

[0072] The memory stores computer-executed instructions;

[0073] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0074] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0075] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed, implements the first aspect and / or various possible implementations of the first aspect.

[0076] The substation inspection method, apparatus, equipment, and medium provided in this application embodiment first receive natural language instructions input by the user, including the inspection object and inspection requirements of the inspection task. Then, a large language model is used to parse the natural language instructions to generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for inspecting the inspection object; a second query sub-task for querying the model information of the recognition model for executing the inspection requirement; and an execution sub-task for executing the inspection processing. Then, each sub-task in the sub-task sequence is executed to acquire an image including the inspection object through the inspection equipment, and to recognize the image through the recognition model to obtain the inspection result corresponding to the inspection task. Finally, the inspection result is output. Based on a large language model, this method can quickly and flexibly decompose and execute temporary and specific inspection requirements input by the user, automatically inspect the corresponding inspection object, realize a rapid response to user needs, and improve the intelligence level of substation inspection. Attached Figure Description

[0077] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0078] Figure 1 This is a flowchart illustrating a substation inspection method provided in Embodiment 1 of this application.

[0079] Figure 2 This is a flowchart illustrating a substation inspection method provided in Embodiment 2 of this application.

[0080] Figure 3 This is a schematic diagram of the substation inspection device provided in Embodiment 4 of this application;

[0081] Figure 4 This is a schematic diagram of the substation inspection device provided in Embodiment 5 of this application;

[0082] Figure 5 A schematic diagram of the structure of the computer device provided in this application.

[0083] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0084] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0085] To address the technical problems mentioned above, the inventors, while researching substation inspection methods, discovered that a large language model can be used to parse natural language commands input by users to instruct on inspection tasks. The user's natural language commands are decomposed into multiple executable sub-tasks. Based on the principle of querying before execution, when executing these sub-tasks, the corresponding inspection task is performed based on the information obtained from the initial query, thereby obtaining the corresponding inspection results. Based on this, a rapid and flexible response can be provided to temporary and specific inspection needs input by users, thus improving the level of intelligence in substation inspection.

[0086] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0087] Figure 1 This is a flowchart illustrating a substation inspection method provided in Embodiment 1 of this application, as shown below. Figure 1 As shown, the method includes:

[0088] S101. Receive natural language instructions input by the user, wherein the natural language instructions include the inspection objects and inspection requirements of the inspection task.

[0089] In this solution, users can input natural language commands to instruct inspection tasks on their terminal devices. The server receives these commands, parses them using a large language model, and generates a sequence of subtasks. When executing these subtasks, the server first queries the equipment information of the inspection device and the model information of the recognition model. Based on the retrieved equipment information, the server remotely controls the inspection device to acquire images of the inspection objects. Then, based on the retrieved model information, it calls the recognition model to recognize the images, thus obtaining the inspection results. Finally, the inspection results are output to the terminal device.

[0090] In practical applications, users input natural language commands through an interactive interface on a terminal device. These commands can be input in the form of text or voice. The terminal device can be a handheld terminal (such as a mobile phone or tablet), a computer, or a dedicated maintenance terminal. The interactive interface can be a graphical interface such as a web page or client software. For example, a user inputs natural language commands into the interactive interface of a computer's client software.

[0091] In addition, the server can be deployed either at the edge or in the cloud.

[0092] Inspection equipment can be edge devices used for inspecting power equipment, such as inspection robots with image acquisition and power supply or fixed-position cameras with pan-tilt control functions (such as cameras from brands like Dahua and Hikvision).

[0093] In this step, the inspection target of the inspection task can be indicated by one or more of the following: location area, type of power equipment, and unique identifier of power equipment. This indicates at least one power equipment that needs to be inspected. For example, the inspection target can be an insulator in area A, or a main transformer with the number CN-2023-001, or a circuit breaker with model A ZF12-126 / 3150-40, etc. The inspection requirements of the inspection task refer to the inspection content of the inspection target. For example, the inspection requirements can be weed detection, oil leak detection, temperature detection, reading detection, etc.

[0094] For example, natural language commands could be "Check whether there is oil leakage in the main transformer body with number CN-2023-001" or "Check whether the meter readings in area A are normal".

[0095] S102. The natural language instructions are parsed using a large language model to generate a sequence of subtasks corresponding to the inspection task. The subtask sequence includes: a first query subtask for querying the equipment information of the inspection equipment for the inspection object; a second query subtask for querying the model information of the recognition model for executing the inspection requirement; and an execution subtask for performing the inspection processing.

[0096] In this step, a Large Language Model (LLM) is used to parse the natural language instruction in order to understand the inspection object (i.e., to identify at least one power device that needs to be inspected) and the inspection requirements (i.e., to identify the inspection items for the at least one power device). Then, based on the understood inspection object and inspection requirements, a sequence of subtasks corresponding to the inspection task is generated.

[0097] The subtask sequence includes query subtasks for retrieving information and execution subtasks for performing inspection processes. The query subtasks include a first query subtask for retrieving equipment information of the inspection equipment for the inspection object and a second query subtask for retrieving model information of the identification model for executing the inspection requirement.

[0098] It should be understood that the first query subtask and the second query subtask are used to query the equipment information and model information corresponding to the inspection task, respectively, to provide an information basis for executing subsequent execution subtasks. Specifically, the equipment information is used to generate corresponding inspection equipment control instructions based on the equipment information when executing subsequent execution subtasks, so as to control the inspection equipment to acquire an image including the inspection object; the model information is used to generate corresponding recognition model invocation instructions based on the equipment information when executing subsequent execution subtasks, so as to invoke the recognition model to recognize the image.

[0099] For example, if the natural language instruction is "Check whether there is oil leakage in the main transformer body of area A", the corresponding generated subtask sequence is as follows: "1. Query the equipment information of the inspection equipment associated with the main transformer of area A; 2. Query the model information of the recognition model that can identify oil leakage in the transformer; 3. Generate a control instruction based on the queried equipment information, and execute the control instruction to control the inspection equipment to collect images including the inspection object; 4. Generate a call instruction based on the queried model information, and execute the call instruction to call the recognition model to recognize the image and obtain the recognition result; 5. Process the recognition result to obtain the inspection result."

[0100] In practical implementation, the large language model can be a closed-source large language model or an open-source large language model that has been pre-trained. For example, training the large language model can involve: acquiring multiple natural language instructions and manually annotating the sub-task sequences corresponding to each instruction to obtain multiple instruction-task pairs; training the pre-initialized large language model based on these instruction-task pairs to obtain a natural language model used to parse natural language instructions and generate sub-task sequences. The initial large language model can be a Transformer model, or a Transformer (Bidirectional and Auto-Regressive Transformer, BART) model that combines bidirectional language modeling and autoregressive mechanisms, etc.

[0101] In one possible implementation, a retrieval-augmented generation (RAG) mechanism can be employed. This allows the large language model to incorporate substation information from a knowledge base when processing natural language instructions, enhancing its understanding of the instructions and assisting in generating more accurate subtask sequences. The knowledge base may include the equipment type, unique identifier, and physical location of each power device in the substation. Optionally, the knowledge base may include substation information such as electronic maps and substation area divisions. This substation information can help the large language model identify at least one power device represented by the inspection object in the natural language instruction.

[0102] S103. Execute each subtask in the subtask sequence to obtain an image including the inspection object through the inspection equipment, and recognize the image through the recognition model to obtain the inspection result corresponding to the inspection task.

[0103] In this step, the first query task is executed to obtain the equipment information of the inspection equipment for the inspection object. At the same time, the second query task is executed to obtain the model information of the identification model for querying and executing the inspection requirement. Then, based on the equipment information and the model information, the execution sub-task is executed to obtain the inspection result of the inspection task.

[0104] In practical applications, the inspection equipment can also acquire videos of the inspected objects, and correspondingly, the recognition model can also perform recognition based on videos.

[0105] The inspection results refer to the inspection results after the inspection object has undergone the corresponding requirement inspection, which are used to indicate whether the inspection object has any corresponding problems.

[0106] S104, Output inspection results.

[0107] In this step, the inspection results need to be output to the terminal for users to view.

[0108] For example, users can view the inspection results in the corresponding interactive interface within the terminal's graphical interface.

[0109] The substation inspection method provided in this embodiment first receives natural language instructions from the user, including the inspection object and inspection requirements. Then, it uses a large language model to parse the natural language instructions and generate a sub-task sequence corresponding to the inspection task. This sub-task sequence includes: a first query sub-task for querying equipment information of the inspection equipment for the inspection object; a second query sub-task for querying model information of the recognition model for executing the inspection requirement; and an execution sub-task for executing the inspection process. Then, it executes each sub-task in the sub-task sequence to acquire an image of the inspection object through the inspection equipment, recognize the image through the recognition model, obtain the inspection result corresponding to the inspection task, and finally output the inspection result. Based on a large language model, this method can quickly and flexibly decompose and execute temporary and specific inspection requirements input by the user, automatically inspect the corresponding inspection object, achieve rapid response to user needs, and improve the intelligence level of substation inspection.

[0110] Figure 2 This is a flowchart illustrating a substation inspection method provided in Embodiment 2 of this application, as shown below. Figure 2 As shown, based on the above embodiments, this embodiment executes subtasks including a control device task, a model invocation task, and a result analysis task. Accordingly, this embodiment provides a detailed description of the specific method for "executing each subtask in the subtask sequence to acquire images including the inspection objects through the inspection device, and recognizing the images through a recognition model to obtain the inspection results corresponding to the inspection task," including:

[0111] S201. Execute the first query subtask to obtain the equipment information of the inspection equipment, including the control interface information of the inspection equipment and the preset control parameters.

[0112] In this step, the first query subtask will be executed to query the inspection equipment that can be used to inspect the inspection object, and to obtain equipment information including the control interface information of the inspection equipment and the preset control parameters.

[0113] Among them, the control interface information refers to the interface data required when remotely controlling the inspection equipment; the preset control parameters refer to the optimal control parameters preset when using the inspection equipment to collect data from the inspection object. Based on the control parameters, the inspection equipment can be controlled to align with the inspection object and acquire images including the inspection object. For example, the preset control parameters may include the rotation angle and focal length of a fixed-position camera.

[0114] In practical applications, to avoid confusion caused by different brands and models of inspection equipment having different control protocols, the control interface information of the inspection equipment can be based on the relevant data of the standardized Application Programming Interface (API) corresponding to that inspection equipment. Based on the relevant data of the standardized API corresponding to the inspection equipment, the inspection object can be invoked and controlled.

[0115] In one possible implementation, the first query subtask can be executed in the following manner to obtain the equipment information of the inspection equipment.

[0116] Step 1: Use a large language model to generate the first query condition based on the inspection object.

[0117] In this step, a large language model is used to generate first query conditions that the knowledge base can recognize, based on the characteristics of each power device represented by the inspection object (such as at least one of the physical location, type or unique identifier of the power device). These conditions are used to query the inspection equipment corresponding to each power device represented by the inspection object.

[0118] Step 2: Based on the first query condition, retrieve the equipment information of the inspection equipment for the inspection object from the knowledge base.

[0119] The knowledge base includes the equipment type, unique identifier, and physical location of each power device in the substation, as well as the inspection range, control interface information, and preset control parameters of each inspection device.

[0120] It should be noted that the knowledge base used here can be the same knowledge base used when parsing natural language instructions and generating subtask sequences using a large language model.

[0121] In this step, based on the equipment type, unique identifier, and physical location of each power device in the knowledge base, and the inspectable range of each inspection device, it is necessary to query the knowledge base to find the inspection device that matches the first query condition and can inspect the inspection object, and then obtain the control interface information and preset control parameters corresponding to the inspection device from the knowledge base.

[0122] This implementation method uses a large language model to generate the first query condition based on the inspection object. Based on the first query condition, it retrieves the equipment information of the inspection equipment that is inspecting the inspection object from the knowledge base. This accurately obtains the equipment information that matches the inspection object, providing a foundation for subsequent image acquisition and ensuring the execution of the inspection task.

[0123] In one possible implementation, RAG can be used to enable the large language model to combine information from the knowledge base and query the knowledge base to obtain the equipment information of the inspection equipment for the inspection object.

[0124] S202. Execute the second query subtask to obtain the model information of the identification model, wherein the model information includes the calling interface information of the identification model.

[0125] In this step, the second query subtask will be executed to retrieve the identification model that can be used to perform the inspection requirement, and to obtain model information, including the identification model's API call information. The identification model's API call information refers to the interface data required to call the identification model.

[0126] In one possible implementation, the API call information for the recognition model can be related data from a standard API corresponding to the recognition model. Based on this standardized API data, the recognition model can be invoked.

[0127] In one possible implementation, the second query subtask can be executed in the following manner to obtain the model information of the identification model:

[0128] Step 1: Use a large language model to generate the second query conditions based on the inspection requirements.

[0129] In this step, a large language model is needed to transform the inspection requirements into query conditions that the image recognition model can recognize in the management table, thus obtaining the second query condition.

[0130] Step 2: Based on the second query condition, retrieve the model information of the recognition model that performs the inspection requirement from the pre-established image recognition model management table.

[0131] The image recognition model management table includes the function of each recognition model and its API call information. It should be understood that the recognition models in the image recognition model management table can cover general visual language models and various artificial intelligence image recognition models specific to power equipment (such as meter reading models, switch status recognition models, infrared temperature measurement analysis models, insulator detection models, oil leakage detection models, smoke and fire detection models, etc.).

[0132] It should be noted that each recognition model has image processing capabilities, enabling it to process the input image and output the recognition result.

[0133] For example, the functions of the identification model include meter reading, transformer oil leakage detection, switch status identification, infrared thermometry, insulator damage detection, and smoke and fire detection.

[0134] In this step, based on the function of each recognition model in the image recognition model management table, it is necessary to query the knowledge base to find the recognition model that matches the first query condition and matches the second query condition, which can execute the inspection requirement, and then obtain the call corresponding to the inspection requirement from the image recognition management table.

[0135] This implementation method uses a large language model to generate a second query condition based on the inspection requirements. Based on the second query condition, it retrieves the model information of the recognition model that performs the inspection requirement from a pre-established image recognition model management table. This accurately obtains the model information that matches the inspection requirements, providing a foundation for subsequent image recognition and ensuring the execution of the inspection task.

[0136] S203. Based on the control interface information and control parameters, execute the control equipment task to control the inspection equipment to perform image acquisition and obtain images including the inspection objects.

[0137] In this step, based on the control interface information and preset control parameters, the control equipment task needs to be executed to control the inspection equipment to acquire images, including the inspection object.

[0138] Specifically, the tasks of executing the control equipment include: using a large language model to generate control instructions for the inspection equipment based on the control interface information and control parameters; executing the control instructions to control the inspection equipment to perform image acquisition and obtain an image including the inspection object.

[0139] In one possible implementation, when performing control device tasks, image quality can be detected, and if the image quality does not meet preset requirements, a large language model can be used to supplement and generate fine-tuning instructions, and then the control parameters can be adjusted according to the fine-tuning instructions.

[0140] Specifically, to ensure the high quality of the images used for recognition, thereby guaranteeing the accuracy of the recognition model's results, image quality checks can be performed after image acquisition (e.g., checking image brightness, sharpness, and resolution; the presence of inspection objects in the image; and the integrity of the inspection objects in the image) to determine whether the image quality meets preset requirements.

[0141] For example, the Laplacian operator can be used to calculate the gradient value of an image to detect the sharpness of the image; or a target detection model (such as the YOLO series models) can be used to detect whether the inspected object is completely present in the image.

[0142] Furthermore, when the image quality does not meet the preset requirements, a large language model can be used to generate fine-tuning instructions, and then the control parameters can be adjusted according to the fine-tuning instructions. Based on the adjusted control parameters, a higher quality image can be obtained again.

[0143] For example, fine-tuning instructions for camera focal length parameters can be generated using a large language model based on the sharpness detection results; fine-tuning instructions for camera rotation angle parameters can be generated using a large language model based on the detection results of whether the inspected object is completely present in the image.

[0144] This implementation method effectively ensures the quality of the images used for recognition and improves the accuracy of recognition results by performing image quality detection when the image quality does not meet the preset requirements, using a large language model to generate fine-tuning instructions, and then adjusting the control parameters according to the fine-tuning instructions. This improves the reliability of the inspection results.

[0145] S204. Based on the interface call information, the execution model calls the task control recognition model to recognize the image and obtain the recognition result.

[0146] In this step, based on the API call information, a model call task is executed to control the recognition model to recognize the image and obtain the recognition result. The recognition result refers to the output of the recognition model after processing the image.

[0147] For example, if the recognition model is a meter reading recognition model, the output result is, for example, a numerical value and a confidence level; if the recognition model is a transformer crack recognition model, the output result is, for example, whether there is a crack on the transformer, and when there is a crack, the detection box used to indicate the location of the crack and the confidence level; if the recognition model is a debris recognition model on a utility pole, the output result is, for example, whether there is debris on the utility pole, and when there is debris, the type of debris and the confidence level.

[0148] S205. The execution result analysis task analyzes the identification results to obtain the inspection results corresponding to the inspection task.

[0149] In this step, after obtaining the identification results, a result analysis task will be performed to analyze the identification results and convert them into inspection results that are easy for users to view.

[0150] In this embodiment, the execution of subtasks includes a control device task, a model invocation task, and a result analysis task. When executing these subtasks, a first query subtask is executed to obtain equipment information for the inspection equipment, including its control interface information and preset control parameters. A second query subtask is then executed to obtain the recognition model, including its invocation interface information. Based on the control interface information and the control parameters, the control device task is executed to control the inspection equipment to perform image acquisition, obtaining an image including the inspection object. Next, based on the invocation interface information, the model invocation task is executed to control the recognition model to recognize the image, obtaining the recognition result. Finally, the result analysis task is executed to analyze the recognition result, obtaining the inspection result corresponding to the inspection task. This method, based on the previously queried equipment and model information corresponding to the inspection task, quickly controls the corresponding inspection equipment and locates the corresponding recognition model, improving the efficiency of the inspection task execution.

[0151] This application provides a substation inspection method in its third embodiment. The solution provided in this embodiment introduces a specific method for "analyzing the identification results by performing an execution result analysis task to obtain the inspection results corresponding to the inspection task," including:

[0152] Step 1: Receive natural language instructions input by the user, including the inspection objects and inspection requirements of the inspection task.

[0153] Step 2: Use a large language model to parse the natural language instructions and generate a sequence of subtasks corresponding to the task.

[0154] The subtask sequence includes: a first query subtask for querying equipment information of the inspection equipment for the inspection object; a second query subtask for querying model information of the identification model for executing the inspection requirement; a third query subtask for querying result interpretation rules for interpreting the identification results of the identification model; and an execution subtask for performing the inspection process. The execution subtask includes a control equipment task, a model invocation task, and a result analysis task.

[0155] Compared to the above embodiments, in this embodiment, the subtask sequence generated by the large language model may also include a third query subtask for querying the interpretation rules of the recognition results of the interpretation model.

[0156] Among them, the interpretation result rules refer to the background information of the actual inspection task used to conduct targeted analysis of the output results of the identification model, such as the normal operating standards, normal parameter ranges, historical operating data, and preset anomaly judgment rules of the power equipment represented by the inspection object.

[0157] Step 3: Execute the first query subtask to obtain the equipment information of the inspection equipment. The equipment information includes the control interface information of the inspection equipment and the preset control parameters.

[0158] Step 4: Execute the second query subtask to obtain the model information of the recognition model, wherein the model information includes the calling interface information of the recognition model.

[0159] Step 5: Execute the third query subtask to generate a third query condition based on the inspection object and the inspection requirement using a large language model; based on the third query condition, query the result interpretation rules of the recognition result of the interpretation recognition model from the knowledge base. The knowledge base also includes the result interpretation rules corresponding to different requirements when inspecting different power equipment in the substation.

[0160] In this step, it is also necessary to obtain result interpretation rules that can be used to interpret the recognition results of the recognition model. Specifically, a large language model is used to generate a third query condition that can be recognized by the knowledge base based on each power equipment represented by the inspection object and the inspection requirements. Then, based on the result interpretation rules corresponding to different requirements of different power equipment in the substation stored in the knowledge base, the result interpretation rules that match the third query conditions are retrieved from the knowledge base.

[0161] Step 6: Based on the control interface information and control parameters, execute the control equipment task to control the inspection equipment to perform image acquisition and obtain an image including the inspection object.

[0162] Step 7: Based on the interface call information, execute the model call task to control the recognition model to recognize the image and obtain the recognition result.

[0163] Step 8: Using a large language model based on result interpretation rules, interpret the recognition results of the recognition model to obtain the interpretation results.

[0164] In this step, when performing the result analysis task, a large language model can be used to perform reasoning, judgment and other interpretation operations on the recognition results of the recognition model based on the result interpretation rules, so as to obtain interpretation results that can directly indicate whether the inspection object has corresponding problems.

[0165] For example, if the natural language instruction is "Check if the reading of the oil level gauge in area A is abnormal", the result interpretation rules can include the normal reading range of the oil level gauge and the historical readings of the oil level gauge. Accordingly, based on the normal reading range of the oil level gauge, the readings identified by the recognition model can be analyzed in conjunction with the historical readings of the oil level gauge to determine the historical trend and infer whether the reading of each oil level gauge in area A is abnormal, thus obtaining the interpretation result.

[0166] In one possible implementation, the interpretation result includes the status of each electrical device represented by the inspected object, which includes normal, abnormal, and suspected abnormal status. Specifically, when the large language model analyzes the recognition result in conjunction with the result interpretation rules, if it is impossible to clearly determine the status of any electrical device (e.g., the reading of the identified oil level meter is at the critical value of the normal range), then the status of the electrical device is judged as suspected abnormal.

[0167] Optionally, if, during the result analysis task, the interpretation results indicate that there are power devices in a state of suspected abnormality among the power devices represented by the inspected objects, then the large language model is used to proactively initiate a dialogue with the user (i.e., send a prompt to the user) to prompt the user to conduct further inspection of the power device.

[0168] Step 9: Based on the interpretation results, obtain the inspection results corresponding to the inspection task.

[0169] In this step, to enable users to more intuitively view the processing of the inspection task, the interpretation results will be summarized to obtain the inspection results corresponding to the inspection task.

[0170] In one possible implementation, based on the interpretation results, the inspection results corresponding to the inspection task are obtained, including:

[0171] A large language model is used to generate an analysis report based on natural language instructions, sub-task sequences, images, recognition results, and interpretation results; this analysis report is then used as the inspection result corresponding to the inspection task.

[0172] In this step, the user-input natural language commands, the sub-task sequence generated by the large language model, the images collected by the inspection equipment, the recognition results output by the recognition model, and the interpretation results obtained by the large language model based on the result interpretation rules are summarized to generate an analysis report. This analysis report is then used as the inspection result corresponding to the inspection task.

[0173] It should be understood that returning inspection results to users in the form of analysis reports not only allows users to view the inspection task processing process more intuitively, but also allows them to view relevant evidence (such as images taken on site, and the normal reading range of relevant power equipment, etc.), enabling users to assess the reliability of the interpretation results and further the effectiveness of the inspection task.

[0174] Step 10: Output the inspection results.

[0175] In this step, the inspection results (i.e., the analysis report) are output to the terminal device.

[0176] In another possible implementation, a periodic inspection strategy can be formulated based on natural language instructions input by the user. These instructions should include not only the inspection targets and requirements, but also the inspection date, so that the corresponding inspection task can be executed on the designated date.

[0177] Furthermore, after completing the inspection task and obtaining the inspection results on each inspection date, the inspection results can be automatically pushed to relevant users. For example, the push can be made via mobile phone text message, application message, email, etc.

[0178] In one possible implementation, the knowledge base also records the historical inspection results corresponding to each inspection task. The natural language commands input by the user can also be result query commands, including instructions for the inspection task requiring a result query. Based on this result query command, historical inspection results matching the query command can be retrieved from the knowledge base and returned to the terminal for the user to view.

[0179] The substation inspection method provided in this embodiment obtains the result interpretation rules associated with the inspection task by querying the knowledge base, and interprets the recognition results output by the recognition model based on the actual inspection task scenario according to the result interpretation rules. This not only improves the accuracy of the inspection, but also enriches the content of the inspection results, and further improves the intelligence level of substation inspection.

[0180] Figure 3 This is a schematic diagram of the substation inspection device provided in Embodiment 4 of this application, as shown below. Figure 3 As shown, the substation inspection device 30 provided in this embodiment includes:

[0181] The receiving unit 301 is used to receive natural language instructions input by the user, the natural language instructions including the inspection objects and inspection requirements of the inspection task.

[0182] The generation unit 302 is used to parse the natural language instructions using a large language model and generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for inspecting the inspection object; a second query sub-task for querying the model information of the identification model for executing the inspection requirement; and an execution sub-task for executing the inspection process.

[0183] The execution unit 303 is used to execute each subtask in the subtask sequence to obtain an image including the inspection object through the inspection device, and to identify the image through the recognition model to obtain the inspection result corresponding to the inspection task.

[0184] Output unit 304 is used to output the inspection results.

[0185] The substation inspection device 30 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0186] Figure 4 This is a schematic diagram of the substation inspection device provided in Embodiment 5 of this application, as shown below. Figure 4 As shown, based on the above embodiments, the substation inspection device 40 provided in this embodiment further includes:

[0187] The detection unit is used to perform quality detection on the image when performing control device tasks. If the quality of the image does not meet the preset requirements, it uses the large language model to supplement and generate fine-tuning instructions; and adjusts the control parameters according to the fine-tuning instructions.

[0188] In one possible implementation, the execution subtasks include a device control task, a model invocation task, and a result analysis task; the execution unit 303 includes;

[0189] The first query module is used to execute the first query subtask to obtain the equipment information of the inspection equipment. The equipment information includes the control interface information of the inspection equipment and preset control parameters.

[0190] The second query module is used to execute the second query subtask to obtain the model information of the recognition model, wherein the model information includes the calling interface information of the recognition model;

[0191] The first execution module is used to execute the control device task based on the control interface information and the control parameters to control the inspection device to perform image acquisition and obtain an image including the inspection object;

[0192] The second execution module is used to execute the model calling task based on the calling interface information to control the recognition model to recognize the image and obtain the recognition result;

[0193] The third execution module is used to perform the result analysis task to analyze the identification results and obtain the inspection results corresponding to the inspection task.

[0194] In one possible implementation, the first query module is configured to:

[0195] A large language model is used to generate the first query conditions based on the inspection object;

[0196] Based on the first query condition, retrieve the equipment information of the inspection equipment for the inspection object from the knowledge base;

[0197] The knowledge base includes the equipment type, unique identifier, and physical location of each power device in the substation, as well as the inspection range, control interface information, and preset control parameters of each inspection device.

[0198] In one possible implementation, the second query module is used for:

[0199] The second query condition is generated based on the inspection requirements using the large language model.

[0200] Based on the second query condition, retrieve the model information of the recognition model that performs the inspection requirement from the pre-established image recognition model management table;

[0201] The image recognition model management table includes the function of each recognition model and the calling interface information.

[0202] In one possible implementation, the knowledge base also includes result interpretation rules corresponding to different requirements of inspection of different power equipment in the substation; the subtask sequence also includes a third query subtask for querying the result interpretation rules for interpreting the recognition results of the recognition model.

[0203] Accordingly, the execution unit 303 further includes:

[0204] The third query module is used to execute the third query subtask to generate third query conditions based on the inspection object and the inspection requirements using the large language model.

[0205] Based on the third query condition, query the knowledge base for result interpretation rules that explain the recognition results of the recognition model;

[0206] Accordingly, the third execution module specifically includes:

[0207] An explanation module is used to interpret the recognition results of the recognition model based on the result explanation rules using the large language model, and obtain an explanation result;

[0208] The acquisition module is used to acquire the inspection result corresponding to the inspection task based on the interpretation result.

[0209] In one possible implementation, the acquisition module is configured to:

[0210] An analysis report is generated using the large language model based on the natural language instructions, the subtask sequence, the image, the recognition result, and the interpretation result.

[0211] The analysis report shall be used as the inspection result corresponding to the inspection task.

[0212] The substation inspection device provided in this embodiment can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0213] Figure 5 A schematic diagram of the structure of the computer device provided in this application. Figure 5 As shown, the computer device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the computer device 40 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0214] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0215] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0216] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0217] The memory may include read-only memory and random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0218] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0219] This application also provides a computer program product, including a computer program that, when executed, implements the above-described substation inspection method.

[0220] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, implement the above-mentioned substation inspection method.

[0221] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as SRAM, EEPROM, EPROM, PROM, ROM, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0222] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside within an ASIC. Alternatively, the processor and the readable storage medium can exist as discrete components in a device.

[0223] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0224] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0225] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0226] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0227] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0228] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A substation inspection method, characterized in that, include: Receive natural language instructions input by the user, the natural language instructions including the inspection objects and inspection requirements of the inspection task; The natural language instructions are parsed using a large language model to generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for the inspection object; a second query sub-task for querying the model information of the recognition model for executing the inspection requirement; and an execution sub-task for performing the inspection processing. Each subtask in the subtask sequence is executed to acquire an image including the inspection object through the inspection device, and the image is identified through the recognition model to obtain the inspection result corresponding to the inspection task. Output the inspection results.

2. The method according to claim 1, characterized in that, The execution subtasks include device control tasks, model invocation tasks, and result analysis tasks; The execution of each subtask in the subtask sequence to acquire an image including the inspected object through the inspection device, and to identify the image through the recognition model to obtain the inspection result corresponding to the inspection task includes: Execute the first query subtask to obtain the equipment information of the inspection equipment, which includes the control interface information of the inspection equipment and preset control parameters. Execute the second query subtask to obtain the model information of the recognition model, the model information including the calling interface information of the recognition model; Based on the control interface information and the control parameters, the control device task is executed to control the inspection device to perform image acquisition, thereby obtaining an image including the inspection object; Based on the interface call information, the model call task is executed to control the recognition model to recognize the image and obtain the recognition result; The result analysis task is performed to analyze the identification results and obtain the inspection results corresponding to the inspection task.

3. The method according to claim 2, characterized in that, The execution of the first query subtask to obtain the equipment information of the inspection equipment of the inspection object includes: A large language model is used to generate the first query conditions based on the inspection object; Based on the first query condition, retrieve the equipment information of the inspection equipment for the inspection object from the knowledge base; The knowledge base includes the equipment type, unique identifier, and physical location of each power device in the substation, as well as the inspection range, control interface information, and preset control parameters of each inspection device.

4. The method according to claim 2, characterized in that, The execution of the second query subtask to obtain model information for the identification model that performs the inspection requirement includes: The large language model is used to generate a second query condition based on the inspection requirements; Based on the second query condition, retrieve the model information of the recognition model that performs the inspection requirement from the pre-established image recognition model management table; The image recognition model management table includes the function of each recognition model and the calling interface information.

5. The method according to claim 3, characterized in that, The knowledge base also includes the result interpretation rules corresponding to the different requirements of inspection of different power equipment in the substation; The subtask sequence also includes a third query subtask for querying and interpreting the result interpretation rules of the recognition model; Accordingly, the method further includes: The third query subtask is executed to generate third query conditions based on the inspection object and the inspection requirements using the large language model. Based on the third query condition, query the knowledge base for result interpretation rules that explain the recognition results of the recognition model; Accordingly, the execution of the result analysis task analyzes the identification results to obtain the inspection results corresponding to the inspection task, including: The recognition results of the recognition model are interpreted using the large language model based on the result interpretation rules to obtain the interpretation results; Based on the interpretation results, the inspection results corresponding to the inspection task are obtained.

6. The method according to claim 5, characterized in that, The step of obtaining the inspection result corresponding to the inspection task based on the interpretation result includes: An analysis report is generated using the large language model based on the natural language instructions, the subtask sequence, the image, the recognition result, and the interpretation result. The analysis report shall be used as the inspection result corresponding to the inspection task.

7. The method according to claim 2, characterized in that, The method further includes: When performing control device tasks, the image is subjected to quality detection. If the quality of the image does not meet the preset requirements, the large language model is used to supplement and generate fine-tuning instructions. The control parameters are adjusted according to the fine-tuning instructions.

8. A substation inspection device, characterized in that, include: The receiving unit is used to receive natural language instructions input by the user, the natural language instructions including the inspection objects and inspection requirements of the inspection task. The generation unit is used to parse the natural language instructions using a large language model and generate a sub-task sequence corresponding to the inspection task. The sub-task sequence includes: a first query sub-task for querying the equipment information of the inspection equipment for inspecting the inspection object; a second query sub-task for querying the model information of the recognition model for executing the inspection requirement; and an execution sub-task for executing the inspection process. An execution unit is used to execute each subtask in the subtask sequence to acquire an image including the inspection object through the inspection device, and to identify the image through the recognition model to obtain the inspection result corresponding to the inspection task. The output unit is used to output the inspection results.

9. A computer device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the method as described in any one of claims 1-7.

Images