Patrol image processing method, device, equipment and medium

Abstract

The invention discloses an inspection image processing method and device, equipment and a medium. The method comprises the following steps: after a current inspection image of a target power line is received, determining a category label of an object in the current inspection image through a target detection model; if the category label contains the key power equipment and / or personnel, cutting out an image area containing the key power equipment and / or personnel in the current inspection image, and determining an equipment reading text and / or a personnel behavior description text through an instance segmentation model and / or a behavior recognition model; determining a feature text of the current inspection image; and inputting a feature text processing prompt instruction corresponding to the feature text into a pre-trained large language model to obtain abnormal condition description information. According to the embodiment of the invention, the image shot in the inspection process of the power line can be automatically processed, and the information used for describing whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects or not is generated.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an inspection image processing method, device, equipment and medium. Background Art

[0002] In order to ensure the stable operation of the power system, it is necessary to inspect each power line in the power system. During the inspection of the power line, the drone is usually controlled to fly along the power line, and the image acquisition device is used to continuously shoot the power line during the flight, and the captured images are transmitted back to the designated electronic device in real time. After receiving the image taken by the drone, the designated electronic device needs to process the image to determine whether the image contains abnormal key power equipment, abnormal personnel, and abnormal objects.

[0003] In the related art, the commonly used inspection image processing scheme is: after the designated electronic device receives the image taken by the drone, it sends the image taken by the drone to the technician. The technician processes the image taken by the drone to determine whether the image contains abnormal critical power equipment, abnormal personnel and abnormal objects, and generates information used to describe whether the image contains abnormal critical power equipment, abnormal personnel and abnormal objects. The inspection image processing scheme in the related art requires manual operation to process the images taken by the drone during the inspection of the power line. It is impossible to automatically process the images taken by the drone during the inspection of the power line to generate information used to describe whether the image contains abnormal critical power equipment, abnormal personnel and abnormal objects. The labor cost and time cost are high, the processing efficiency is low, and the accuracy is difficult to guarantee. Summary of the invention

[0004] The present invention provides an inspection image processing method, device, equipment and medium to solve the problem that the inspection image processing scheme in the related art needs to process the images taken by the drone during the inspection of the power line through manual operation, and cannot automatically process the images taken by the drone during the inspection of the power line to generate information for describing whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects, the labor cost and time cost are high, the processing efficiency is low, and the accuracy is difficult to guarantee.

[0005] According to one aspect of the present invention, there is provided a method for processing inspection images, comprising:

[0006] After receiving the current inspection image of the target power line, performing target detection on the current inspection image by using a pre-trained target detection model to determine the category label of the object in the current inspection image;

[0007] Determining whether the category tag includes critical power equipment and / or personnel;

[0008] If the category label includes critical power equipment and / or personnel, the image area including the critical power equipment and / or personnel in the current inspection image is cropped out to obtain a cropped image of the critical power equipment and / or a cropped image of the personnel, and image recognition is performed on the cropped image of the critical power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text;

[0009] Determine the device reading text and / or the personnel behavior description text and the category label as feature text of the current inspection image;

[0010] The feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and the image description information is provided to a target user; wherein the image description information is used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel and abnormal objects.

[0011] According to another aspect of the present invention, there is provided an inspection image processing device, comprising:

[0012] A target detection module is used to, after receiving a current inspection image of a target power line, perform target detection on the current inspection image through a pre-trained target detection model to determine a category label of an object in the current inspection image;

[0013] A tag processing module, used to determine whether the category tag contains key power equipment and / or personnel;

[0014] a recognition module, configured to, if the category label includes key power equipment and / or personnel, crop the image region including the key power equipment and / or personnel in the current inspection image to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and perform image recognition on the cropped image of the key power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text;

[0015] A feature text determination module, used to determine the device reading text and / or the personnel behavior description text and the category label as the feature text of the current inspection image;

[0016] A description information determination module is used to input the feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, obtain the image description information of the current inspection image output by the large language model, and provide the image description information to a target user; wherein the image description information is information used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel and abnormal objects.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor;

[0019] and a memory communicatively coupled to the at least one processor;

[0020] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the inspection image processing method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection image processing method described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention is to perform target detection on the current inspection image through a pre-trained target detection model after receiving the current inspection image of the target power line, and determine the category label of the object in the current inspection image; then determine whether the category label contains key power equipment and / or personnel; if the category label contains key power equipment and / or personnel, crop the image area containing the key power equipment and / or personnel in the current inspection image to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and crop the image of the key power equipment through a pre-trained instance segmentation model and / or a behavior recognition model. The image and / or personnel cropped image are used for image recognition to determine the equipment reading text and / or personnel behavior description text; the equipment reading text and / or personnel behavior description text and the category label are determined as the feature text of the current inspection image; the feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into the pre-trained large language model to obtain the image description information of the current inspection image output by the large language model, and the image description information is provided to the target user. The image description information is used to describe whether the current inspection image contains abnormal key power equipment, abnormal personnel and abnormal objects, which solves the inspection problem in the related technology. The inspection image processing scheme requires manual operation to process the images taken by the drone during the inspection of the power line. It is impossible to automatically process the images taken by the drone during the inspection of the power line to generate information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects. The labor cost and time cost are high, the processing efficiency is low, and the accuracy is difficult to guarantee. The images taken by the drone during the inspection of the power line can be automatically processed through the pre-trained target detection model, instance segmentation model and behavior recognition model to determine the feature text used to characterize the objects contained in the image. Then, based on the feature text of the image and the pre-trained large language model, the image description information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately obtained, and the image description information is provided to the target user, so that the images taken by the drone during the inspection of the power line are automatically processed during the inspection of the power line, and the information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately generated, which reduces the labor cost and time cost of the inspection image processing process and improves the processing efficiency and accuracy of the inspection image processing process.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 The present invention provides a flowchart of a method for processing inspection images.

[0026] Figure 2 The present invention provides a flowchart of a method for processing inspection images according to a second embodiment of the present invention.

[0027] Figure 3 A schematic diagram of the structure of an inspection image processing device provided in Embodiment 3 of the present invention.

[0028] Figure 4 A schematic diagram of the structure of an electronic device for implementing the inspection image processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "target", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise", "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1This is a flowchart of a patrol image processing method provided in the first embodiment of the present invention. This embodiment is applicable to the case where images taken by a drone during the patrol inspection of a power line are processed. The method can be executed by a patrol image processing device, which can be implemented in the form of hardware and / or software, and the patrol image processing device can be configured in an electronic device. The electronic device is used to process images taken by a drone during the patrol inspection of a power line. Figure 1 As shown, the method includes:

[0033] Step 101: After receiving a current inspection image of a target power line, target detection is performed on the current inspection image using a pre-trained target detection model to determine a category label of an object in the current inspection image.

[0034] Optionally, in general, the power system includes multiple power lines. The power line may be a line used to transmit electric energy between a power plant, a substation and a power user. During the operation of the power system, it is necessary to inspect each power line in the power system. During the inspection of the power line, a drone used to inspect the power line will fly along the power line, and continuously shoot the power line through an image acquisition device during the flight, and send the shot images to the electronic device in real time. After receiving the image shot by the drone, the electronic device needs to determine whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects. The abnormal key power equipment may be a key power equipment with an abnormal state. The key power equipment may be an electric energy meter. The key power equipment with an abnormal state may be an electric energy meter whose text for representing the voltage, current and electric quantity value displayed is not within a preset value range. The preset value range may be the value range in which the text for representing the voltage, current and electric quantity value displayed by the key power equipment in a normal state should be. The abnormal person may be a person with irregular behavior. The person with irregular behavior may refer to a person who smokes, does not wear a helmet and / or does not wear rubber gloves. The abnormal object may include trees and bird nests.

[0035] Optionally, the target power line is a power line in the process of inspection. The drone used to inspect the target power line flies along the target power line, and continuously photographs the target power line through an image acquisition device during the flight, and transmits the photographed images back to the electronic device in real time. The current inspection image of the target power line is an image sent by the drone used to inspect the target power line and received by the electronic device at the current moment.

[0036] Optionally, the category label of the object in the image may refer to the category label of each object contained in the image. The objects that may be contained in the image taken by the drone during the inspection process include: the transmission line set in the power line, the pole tower set in the power line, the key power equipment set in the power line, the personnel located around the power line, the trees located on the power line, and the bird's nest located on the power line. The category label of the object may be a word used to characterize the object. The category label of the transmission line set in the power line is the transmission line. The category label of the pole tower set in the power line is the pole tower. The category label of the key power equipment set in the power line is the key power equipment. The category label of the personnel located around the power line is the personnel. The category label of the tree located on the power line is the tree. The category label of the bird's nest located on the power line is the bird's nest. Exemplarily, the objects contained in the image taken by the drone during the inspection process are the transmission line set in the power line, the key power equipment set in the power line, and the personnel located around the power line, and the category labels of the objects in the image taken by the drone during the inspection process are: transmission line, key power equipment, personnel.

[0037] Optionally, the current inspection image is an image taken by a drone used to inspect the target power line during the inspection process. The category label of the object in the current inspection image may refer to the category label of each object contained in the current inspection image. The category label of the object in the current inspection image is a text that can be used to characterize the object contained in the current inspection image.

[0038] Optionally, target detection is performed on the current inspection image through a pre-trained target detection model to determine the category label of the object in the current inspection image, including: inputting the current inspection image into the pre-trained target detection model to obtain the category label of the object in the current inspection image output by the target detection model.

[0039] Optionally, a pre-trained target detection model is provided in the electronic device. The pre-trained target detection model is used to perform target detection on images taken by the drone during the inspection process, and obtain the category labels of objects in the images taken by the drone during the inspection process. The input of the target detection model is the image taken by the drone during the inspection process, and the output of the target detection model is the category labels of objects in the images taken by the drone during the inspection process. The electronic device inputs the current inspection image into the pre-trained target detection model, and the pre-trained target detection model analyzes the current inspection image, determines the category labels of the objects in the current inspection image, and then outputs the category labels of the objects in the current inspection image.

[0040] Optionally, technicians can use a set number of images taken by drones during the inspection process and the category labels of objects in the images taken by each drone during the inspection process to train the machine learning model to obtain a target detection model, and then set the target detection model in the electronic device. The machine learning model can be a neural network model.

[0041] Step 102: Determine whether the category tag includes key power equipment and / or personnel.

[0042] Optionally, after obtaining the category label of the object in the current inspection image output by the target detection model, it is determined whether the category label of the object in the current inspection image includes key power equipment and / or personnel.

[0043] Step 103: If the category label includes critical power equipment and / or personnel, the image area including the critical power equipment and / or personnel in the current inspection image is cropped out to obtain a cropped image of the critical power equipment and / or a cropped image of the personnel, and image recognition is performed on the cropped image of the critical power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text.

[0044] Optionally, the device reading text may be text for representing voltage, current and power value displayed on key power devices disposed in the power lines contained in the image.

[0045] Optionally, the personnel behavior description text may be text used to describe the behavior of personnel located around the power lines contained in the image. The personnel behavior description text may be that the personnel is not smoking and is wearing a safety helmet and rubber gloves. The personnel behavior description text may also be that the personnel is not smoking, is wearing a safety helmet, and is not wearing rubber gloves. The personnel behavior description text may also be that the personnel is not smoking, is not wearing a safety helmet, and is wearing rubber gloves. The personnel behavior description text may also be that the personnel is not smoking, is not wearing a safety helmet and is wearing rubber gloves. The personnel behavior description text may also be that the personnel is smoking, is wearing a safety helmet and rubber gloves. The personnel behavior description text may also be that the personnel is smoking, is wearing a safety helmet, and is not wearing rubber gloves. The personnel behavior description text may also be that the personnel is smoking, is not wearing a safety helmet, and is wearing rubber gloves. The personnel behavior description text may also be that the personnel is smoking, is not wearing a safety helmet, and is wearing rubber gloves. The personnel behavior description text may also be that the personnel is smoking, is not wearing a safety helmet, and is wearing rubber gloves.

[0046] Optionally, if the category labels of objects in the current inspection image include critical power equipment and / or personnel, indicating that the current inspection image contains critical power equipment installed in the power lines and / or personnel located around the power lines, the image area containing the critical power equipment and / or personnel in the current inspection image can be cropped out to obtain a cropped image of the critical power equipment and / or a cropped image of the personnel, and image recognition can be performed on the cropped image of the critical power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or behavior recognition model to determine the equipment reading text and / or the personnel behavior description text.

[0047] Optionally, if the category label includes critical power equipment and / or personnel, the image area including the critical power equipment and / or personnel in the current inspection image is cropped out to obtain a cropped image of critical power equipment and / or a cropped image of personnel, and image recognition is performed on the cropped image of critical power equipment and / or the cropped image of personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text, including: if the category label includes critical power equipment and personnel, the image area including the critical power equipment in the current inspection image is cropped out to obtain a cropped image of critical power equipment, and the cropped image of critical power equipment is input into a pre-trained instance segmentation model to obtain the equipment reading text of the cropped image of critical power equipment output by the instance segmentation model; the image area including personnel in the current inspection image is cropped out to obtain a cropped image of personnel, and the cropped image of personnel is input into a pre-trained behavior recognition model to obtain the personnel behavior description text of the cropped image of personnel output by the behavior recognition model.

[0048] Optionally, the cropped image of key power equipment is an image region containing key power equipment in the current inspection image. The cropped image of personnel is an image region containing personnel in the current inspection image. Optionally, if the category label includes key power equipment and personnel, it indicates that the current inspection image contains key power equipment set in the power line and personnel located around the power line, and the cropped image of key power equipment and the cropped image of personnel can be cropped out. The electronic device can crop out the image region containing key power equipment in the current inspection image through the device image cropping component to obtain a cropped image of key power equipment. The electronic device can crop out the image region containing personnel in the current inspection image through the personnel image cropping component to obtain a cropped image of personnel. The device image cropping component can be a component set in the electronic device for cropping out the image region containing key power equipment in the image. The personnel image cropping component can be a component set in the electronic device for cropping out the image region containing personnel in the image.

[0049] Optionally, a pre-trained instance segmentation model is provided in the electronic device. The pre-trained instance segmentation model is used to perform image recognition on cropped images of key power equipment to obtain the device reading text of the cropped images of key power equipment. The input of the instance segmentation model is the cropped image of key power equipment, and the output of the instance segmentation model is the device reading text of the cropped image of key power equipment. The electronic device inputs the cropped image of key power equipment into the pre-trained instance segmentation model, and the pre-trained instance segmentation model analyzes and recognizes the cropped image of key power equipment, determines the device reading text of the cropped image of key power equipment, and then outputs the device reading text of the cropped image of key power equipment.

[0050] Optionally, the technician can use a set number of cropped images of key power equipment and the device reading text of each cropped image of key power equipment to train the machine learning model to obtain an instance segmentation model, and then set the instance segmentation model in the electronic device. The machine learning model can be a neural network model.

[0051] Optionally, a pre-trained behavior recognition model is provided in the electronic device. The pre-trained behavior recognition model is used to perform image recognition on cropped images of people to obtain a description text of the person's behavior in the cropped images of people. The input of the behavior recognition model is the cropped images of people, and the output of the behavior recognition model is the description text of the person's behavior in the cropped images of people. The electronic device inputs the cropped images of people into the pre-trained behavior recognition model, and the pre-trained behavior recognition model analyzes and recognizes the cropped images of people, determines the description text of the person's behavior in the cropped images of people, and then outputs the description text of the person's behavior in the cropped images of people.

[0052] Optionally, the technician can use a set number of cropped images of people and the description text of the person's behavior in each cropped image to train the machine learning model to obtain a behavior recognition model, and then set the behavior recognition model in the electronic device. The machine learning model can be a neural network model.

[0053] Optionally, if the category label includes critical power equipment but does not include personnel, indicating that the current inspection image includes critical power equipment installed in the power line but does not include personnel located around the power line, then the image area containing the critical power equipment in the current inspection image is cropped out to obtain a cropped image of critical power equipment, and the cropped image of critical power equipment is input into a pre-trained instance segmentation model to obtain the equipment reading text of the cropped image of critical power equipment output by the instance segmentation model.

[0054] Optionally, if the category label does not include critical power equipment but includes personnel, indicating that the current inspection image does not include critical power equipment installed in the power line but includes personnel located around the power line, then the image area containing the personnel in the current inspection image is cropped out to obtain a cropped image of the personnel, and the cropped image of the personnel is input into a pre-trained behavior recognition model to obtain a personnel behavior description text of the cropped image of the personnel output by the behavior recognition model.

[0055] Optionally, if the category label does not include critical power equipment and personnel, it indicates that the current inspection image does not include critical power equipment installed in the power line and personnel located around the power line. There is no need to crop the image area containing critical power equipment and / or personnel in the current inspection image to obtain cropped images of critical power equipment and / or cropped images of personnel, and perform image recognition on the cropped images of critical power equipment and / or cropped images of personnel through a pre-trained instance segmentation model and / or behavior recognition model to determine the equipment reading text and / or personnel behavior description text.

[0056] Step 104: Determine the device reading text and / or the personnel behavior description text and the category label as feature text of the current inspection image.

[0057] Optionally, the feature text of the image taken by the drone during the inspection process may refer to text obtained by processing the image and can be used to characterize the object contained in the image.

[0058] Optionally, if the images taken by the drone during the inspection process include key power equipment installed in the power lines and personnel located around the power lines, the feature text of the image is composed of the category labels of the objects in the image, the equipment reading text of the image, and the personnel behavior description text of the image.

[0059] Optionally, if the image taken by the drone during the inspection process includes key power equipment installed in the power line and does not include personnel located around the power line, the feature text of the image is composed of the category label of the object in the image and the equipment reading text of the image.

[0060] Optionally, if the image taken by the drone during the inspection process does not contain key power equipment installed in the power line, but contains personnel located around the power line, the feature text of the image is composed of the category label of the object in the image and the personnel behavior description text in the image.

[0061] Optionally, if the image taken by the drone during the inspection process does not include key power equipment installed in the power line and personnel located around the power line, the feature text of the image is composed of category labels of objects in the image.

[0062] Optionally, determining the device reading text and / or the personnel behavior description text and the category label as the feature text of the current inspection image includes: determining the device reading text, the personnel behavior description text and the category label as the feature text of the current inspection image.

[0063] Optionally, if the category labels include key power equipment and personnel, indicating that the current inspection image contains key power equipment installed in the power lines and personnel located around the power lines, then after determining the equipment reading text and the personnel behavior description text, the equipment reading text, the personnel behavior description text, and the category labels of the objects in the current inspection image are determined as the feature text of the current inspection image.

[0064] Optionally, if the category label includes key power equipment but does not include personnel, it indicates that the current inspection image includes key power equipment installed in the power line but does not include personnel located around the power line. Then, after determining the equipment reading text, the equipment reading text and the category label of the object in the current inspection image are determined as the feature text of the current inspection image.

[0065] Optionally, if the category label does not include critical power equipment but includes personnel, indicating that the current inspection image does not include critical power equipment installed in the power line but includes personnel located around the power line, then after determining the personnel behavior description text, the personnel behavior description text and the category label of the object in the current inspection image are determined as the feature text of the current inspection image.

[0066] Optionally, if the category label does not include key power equipment and personnel, indicating that the current inspection image does not include key power equipment set in the power line and personnel located around the power line, the category label of the object in the current inspection image is directly determined as the feature text of the current inspection image.

[0067] Optionally, the device reading text and / or the personnel behavior description text and the category label are determined as feature text of the current inspection image, including: determining the device reading text, the personnel behavior description text, the category label and the image description information of the historical inspection image corresponding to the current inspection image as the feature text of the current inspection image.

[0068] Optionally, the historical inspection image corresponding to the current inspection image may be an image taken by the last drone during the inspection process that was processed before the current inspection image was received. The image description information of the historical inspection image corresponding to the current inspection image is text used to describe whether the historical inspection image contains abnormal critical power equipment, abnormal personnel, and abnormal objects. Usually, the objects contained in the historical inspection image corresponding to the current inspection image are the same or similar to the objects contained in the current inspection image. The image description information of the historical inspection image corresponding to the current inspection image is also text that can be used to characterize the objects contained in the current inspection image.

[0069] Optionally, if the category label includes key power equipment and personnel, indicating that the current inspection image contains key power equipment installed in the power line and personnel located around the power line, then after determining the equipment reading text and the personnel behavior description text, the equipment reading text, the personnel behavior description text, the category label of the object in the current inspection image, and the image description information of the historical inspection image corresponding to the current inspection image are determined as the feature text of the current inspection image.

[0070] Optionally, if the category label includes key power equipment but does not include personnel, indicating that the current inspection image includes key power equipment installed in the power line but does not include personnel located around the power line, then after determining the equipment reading text, the equipment reading text, the category label of the object in the current inspection image, and the image description information of the historical inspection image corresponding to the current inspection image are determined as the feature text of the current inspection image.

[0071] Optionally, if the category label does not include critical power equipment but includes personnel, indicating that the current inspection image does not include critical power equipment installed in the power line but includes personnel located around the power line, then after determining the personnel behavior description text, the personnel behavior description text, the category label of the object in the current inspection image, and the image description information of the historical inspection image corresponding to the current inspection image are determined as the feature text of the current inspection image.

[0072] Optionally, if the category label does not include key power equipment and personnel, indicating that the current inspection image does not include key power equipment installed in the power line and personnel located around the power line, the category label of the object in the current inspection image and the image description information of the historical inspection image corresponding to the current inspection image are directly determined as the feature text of the current inspection image.

[0073] Step 105: input the feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, obtain the image description information of the current inspection image output by the large language model, and provide the image description information to the target user.

[0074] The image description information is information used to describe whether the current inspection image contains abnormal key power equipment, abnormal personnel, and abnormal objects.

[0075] Optionally, the pre-trained large language model can be a large-scale language model (LLM) pre-trained by a technician and set in an electronic device for processing text according to prompt instructions. Prompt instructions are text used to instruct the pre-trained large language model to perform specified processing operations on specified text. The prompt instructions are input into the pre-trained large language model, and the pre-trained large language model will perform the specified processing operation on the specified text and output the processing results. Prompt instructions include but are not limited to: feature text processing prompt instructions corresponding to the feature text of the images taken by the drone during the inspection process, and feature text processing prompt instructions corresponding to the inspection question text sent by the user.

[0076] Optionally, the feature text processing prompt instruction corresponding to the feature text of the image taken by the drone during the inspection process is used to instruct the pre-trained large language model to analyze the feature text of the image taken by the drone during the inspection process to generate text of the image description information of the image taken by the drone during the inspection process. The feature text processing prompt instruction corresponding to the feature text of the image taken by the drone during the inspection process contains the feature text of the image taken by the drone during the inspection process. The image description information of the image taken by the drone during the inspection process may be a text used to describe whether the image taken by the drone during the inspection process contains abnormal key power equipment, abnormal personnel and abnormal objects. The image description information of the image may be that the image contains abnormal key power equipment, abnormal personnel and abnormal objects. The image description information of the image may also be that the image does not contain abnormal key power equipment, but contains abnormal personnel and abnormal objects. The image description information of the image may also be that the image does not contain abnormal personnel, but contains abnormal key power equipment and abnormal objects. The image description information of the image may also be that the image does not contain abnormal objects, but contains abnormal key power equipment and abnormal personnel. The image description information of the image may also be that the image does not contain abnormal objects, but contains abnormal key power equipment and abnormal personnel. The image description information of the image may also be that the image does not contain abnormal key power equipment and abnormal personnel, but contains abnormal objects. The image description information of the image may also be that the image does not contain abnormal critical power equipment and abnormal objects, and contains abnormal personnel. The image description information of the image may also be that the image does not contain abnormal personnel and abnormal objects, and contains abnormal critical power equipment. The image description information of the image may also be that the image does not contain abnormal critical power equipment, abnormal personnel, and abnormal objects. The image description information of the image taken by the drone during the inspection process may also be a chart used to describe whether the image taken by the drone during the inspection process contains abnormal critical power equipment, abnormal personnel, and abnormal objects.

[0077] Optionally, the question text processing prompt instruction corresponding to the feature text processing prompt instruction corresponding to the inspection question text sent by the user is a text used to instruct the pre-trained large language model to analyze the inspection question text sent by the user and generate an answer text corresponding to the inspection question text sent by the user. The feature text processing prompt instruction corresponding to the inspection question text sent by the user contains the inspection question text sent by the user. The inspection question text sent by the user is a text used to describe the technical problems found by the user during the inspection of the power line. The answer text corresponding to the inspection question text sent by the user is a text used to describe the answer to the technical problem found by the user during the inspection process.

[0078] Optionally, the feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and the image description information is provided to a target user, including: generating a feature text processing prompt instruction corresponding to the feature text of the current inspection image according to the feature text of the current inspection image and a feature text processing prompt instruction template; inputting the feature text processing prompt instruction into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and providing the image description information to a target user.

[0079] Optionally, the feature text processing prompt instruction template is a feature text processing prompt instruction that does not contain the feature text of the images taken by the drone during the inspection process. The feature text processing prompt instruction template contains a text filling position. The text filling position is a position for filling the feature text of the images taken by the drone during the inspection process for which image description information needs to be generated. After filling the feature text of the images taken by the drone during the inspection process for which image description information needs to be generated into the text filling position in the feature text processing prompt instruction template, a feature text processing prompt instruction corresponding to the feature text of the images taken by the drone during the inspection process for which image description information needs to be generated can be obtained. The feature text processing prompt instruction corresponding to the feature text of the images taken by the drone during the inspection process is used to instruct the pre-trained large language model to analyze the feature text of the images taken by the drone during the inspection process to generate text of the image description information of the images taken by the drone during the inspection process.

[0080] Optionally, based on the feature text of the current inspection image and the feature text processing prompt instruction template, a feature text processing prompt instruction corresponding to the feature text of the current inspection image is generated, including: filling the feature text of the current inspection image into the text filling position in the feature text processing prompt instruction template, thereby obtaining a feature text processing prompt instruction corresponding to the feature text of the current inspection image. The feature text processing prompt instruction corresponding to the feature text of the current inspection image is a text used to instruct a pre-trained large language model to analyze the feature text of the current inspection image and generate image description information of the current inspection image. The image description information of the current inspection image can be a text used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel, and abnormal objects.

[0081] Optionally, the electronic device inputs the feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, and the pre-trained large language model analyzes the feature text of the current inspection image contained in the feature text processing prompt instruction, generates image description information of the current inspection image, and outputs the image description information of the current inspection image. The electronic device obtains the image description information of the current inspection image output by the large language model.

[0082] Optionally, providing the image description information to a target user includes: sending the image description information to a terminal device of the target user. The target user may be a technician responsible for managing the target power line. The terminal device of the target user may be a terminal device used by the target user.

[0083] Optionally, it also includes: after receiving the inspection question text sent by the target user, inputting the question text processing prompt instruction corresponding to the inspection question text into the large language model, obtaining the answer text corresponding to the inspection question text output by the large language model; and sending the answer text to the target user.

[0084] Optionally, the inspection question text sent by the target user is a text used to describe the technical problems found by the target user during the inspection of the target power line. The target user can send the inspection question text to the electronic device through the terminal device. After receiving the inspection question text sent by the target user, the electronic device inputs the question text processing prompt instruction corresponding to the inspection question text into the large language model, obtains the answer text corresponding to the inspection question text output by the large language model, and then sends the answer text to the target user.

[0085] Optionally, a question text processing prompt instruction corresponding to the inspection question text is input into the large language model to obtain an answer text corresponding to the inspection question text output by the large language model, including: generating a question text processing prompt instruction corresponding to the inspection question text based on the inspection question text and a question text processing prompt instruction template; and inputting the question text processing prompt instruction into a pre-trained large language model to obtain an answer text corresponding to the inspection question text output by the large language model.

[0086] Optionally, the question text processing prompt instruction template is a question text processing prompt instruction that does not contain the inspection question text. The question text processing prompt instruction template contains a text filling position. The text filling position is a position for filling the inspection question text. After the inspection question text is filled into the text filling position in the question text processing prompt instruction template, the question text processing prompt instruction corresponding to the inspection question text can be obtained.

[0087] Optionally, based on the inspection question text and the question text processing prompt instruction template, a question text processing prompt instruction corresponding to the inspection question text is generated, including: filling the inspection question text into the text filling position in the question text processing prompt instruction template, thereby obtaining a question text processing prompt instruction corresponding to the inspection question text. The inspection question text is a text used to describe the technical problems found by the target user during the inspection process of the target power line. The question text processing prompt instruction corresponding to the inspection question text is a text used to instruct the pre-trained large language model to generate an answer text corresponding to the inspection question text for the inspection question text. The answer text corresponding to the inspection question text can be a text used to describe the answer to the technical problem found by the target user during the inspection process of the target power line.

[0088] Optionally, the electronic device inputs the question text processing prompt instruction corresponding to the inspection question text into a pre-trained large language model, and the pre-trained large language model analyzes the inspection question text contained in the question text processing prompt instruction, generates an answer text corresponding to the inspection question text, and outputs the answer text corresponding to the inspection question text. The electronic device obtains the answer text corresponding to the inspection question text output by the large language model.

[0089] Optionally, sending the answer text to the target user includes: sending the answer text to a terminal device of the target user.

[0090] Optionally, after the feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into the pre-trained large language model, the pre-trained large language model will generate a code through the Python interpreter that is used to analyze the feature text of the current inspection image contained in the feature text processing prompt instruction and generate the image description information of the current inspection image after running. Then, the generated code is run through the Python interpreter to analyze the feature text of the current inspection image contained in the feature text processing prompt instruction and generate the image description information of the current inspection image. Finally, the image description information of the current inspection image is output. The electronic device obtains the image description information of the current inspection image output by the large language model.

[0091] Optionally, after the question text processing prompt instruction corresponding to the inspection question text is input into the pre-trained large language model, the pre-trained large language model will generate a code through the Python interpreter that is used to analyze the inspection question text contained in the question text processing prompt instruction and generate an answer text corresponding to the inspection question text. Then the generated code is run through the Python interpreter to analyze the inspection question text contained in the question text processing prompt instruction and generate an answer text corresponding to the inspection question text. Finally, the answer text corresponding to the inspection question text is output. The electronic device obtains the answer text corresponding to the inspection question text output by the large language model.

[0092] The technical solution of the embodiment of the present invention is to perform target detection on the current inspection image through a pre-trained target detection model after receiving the current inspection image of the target power line, and determine the category label of the object in the current inspection image; then determine whether the category label contains key power equipment and / or personnel; if the category label contains key power equipment and / or personnel, crop the image area containing the key power equipment and / or personnel in the current inspection image to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and crop the image of the key power equipment through a pre-trained instance segmentation model and / or a behavior recognition model. The image and / or personnel cropped image are used for image recognition to determine the equipment reading text and / or personnel behavior description text; the equipment reading text and / or personnel behavior description text and the category label are determined as the feature text of the current inspection image; the feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into the pre-trained large language model to obtain the image description information of the current inspection image output by the large language model, and the image description information is provided to the target user. The image description information is used to describe whether the current inspection image contains abnormal key power equipment, abnormal personnel and abnormal objects, which solves the inspection problem in the related technology. The inspection image processing scheme requires manual operation to process the images taken by the drone during the inspection of the power line. It is impossible to automatically process the images taken by the drone during the inspection of the power line to generate information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects. The labor cost and time cost are high, the processing efficiency is low, and the accuracy is difficult to guarantee. The images taken by the drone during the inspection of the power line can be automatically processed through the pre-trained target detection model, instance segmentation model and behavior recognition model to determine the feature text used to characterize the objects contained in the image. Then, based on the feature text of the image and the pre-trained large language model, the image description information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately obtained, and the image description information is provided to the target user, so that the images taken by the drone during the inspection of the power line are automatically processed during the inspection of the power line, and the information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately generated, which reduces the labor cost and time cost of the inspection image processing process and improves the processing efficiency and accuracy of the inspection image processing process.

[0093] Embodiment 2

[0094] Figure 2 This is a flowchart of a method for processing inspection images provided in Embodiment 2 of the present invention. This embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0095] Step 201: After receiving a current inspection image of a target power line, the current inspection image is input into a pre-trained target detection model to obtain a category label of an object in the current inspection image output by the target detection model.

[0096] Step 202: Determine whether the category tag includes key power equipment and / or personnel.

[0097] Step 203: If the category label includes critical power equipment and personnel, the image area containing the critical power equipment in the current inspection image is cropped out to obtain a cropped image of the critical power equipment, and the cropped image of the critical power equipment is input into a pre-trained instance segmentation model to obtain the equipment reading text of the cropped image of the critical power equipment output by the instance segmentation model.

[0098] Step 204: crop the image area containing personnel in the current inspection image to obtain a cropped personnel image, and input the cropped personnel image into a pre-trained behavior recognition model to obtain a personnel behavior description text of the cropped personnel image output by the behavior recognition model.

[0099] Step 205: Determine the device reading text, the personnel behavior description text, and the category label as feature text of the current inspection image.

[0100] Step 206: input the feature text processing prompt instruction corresponding to the feature text of the current inspection image into the pre-trained large language model, obtain the image description information of the current inspection image output by the large language model, and provide the image description information to the target user.

[0101] The image description information is information used to describe whether the current inspection image contains abnormal key power equipment, abnormal personnel, and abnormal objects.

[0102] The technical solution of the embodiment of the present invention can automatically process the images taken by the drone during the inspection of the power lines through the pre-trained target detection model, instance segmentation model and behavior recognition model to determine the category labels of the objects in the image, the equipment reading text of the image and the personnel behavior description text of the image. Based on the category labels of the objects in the image, the equipment reading text of the image, the personnel behavior description text of the image and the pre-trained large language model, the image description information for describing whether the image contains abnormal critical power equipment, abnormal personnel and abnormal objects can be accurately obtained, and the image description information can be provided to the target user, so that the images taken by the drone during the inspection of the power lines can be automatically processed during the inspection of the power lines, and the information for describing whether the image contains abnormal critical power equipment, abnormal personnel and abnormal objects can be accurately generated.

[0103] Embodiment 3

[0104] Figure 3 The structure diagram of a patrol image processing device provided by the third embodiment of the present invention is shown in FIG. The device can be configured in an electronic device. Figure 3 As shown, the device includes: a target detection module 301, a label processing module 302, a recognition module 303, a feature text determination module 304 and a description information determination module 305.

[0105] Among them, the target detection module 301 is used to perform target detection on the current inspection image of the target power line through a pre-trained target detection model after receiving the current inspection image, and determine the category label of the object in the current inspection image; the label processing module 302 is used to determine whether the category label contains key power equipment and / or personnel; the recognition module 303 is used to crop the image area containing the key power equipment and / or personnel in the current inspection image if the category label contains the key power equipment and / or personnel, and obtain the cropped image of the key power equipment and / or the cropped image of the personnel, and the cropped image of the key power equipment and / or the cropped image of the personnel is cropped through a pre-trained instance segmentation model and / or a behavior recognition model. The personnel cropped image is subjected to image recognition to determine the equipment reading text and / or the personnel behavior description text; a feature text determination module 304 is used to determine the equipment reading text and / or the personnel behavior description text and the category label as the feature text of the current inspection image; a description information determination module 305 is used to input the feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, obtain the image description information of the current inspection image output by the large language model, and provide the image description information to a target user; wherein the image description information is information used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel, and abnormal objects.

[0106] The technical solution of the embodiment of the present invention is to perform target detection on the current inspection image through a pre-trained target detection model after receiving the current inspection image of the target power line, and determine the category label of the object in the current inspection image; then determine whether the category label contains key power equipment and / or personnel; if the category label contains key power equipment and / or personnel, crop the image area containing the key power equipment and / or personnel in the current inspection image to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and crop the image of the key power equipment through a pre-trained instance segmentation model and / or a behavior recognition model. The image and / or personnel cropped image are used for image recognition to determine the equipment reading text and / or personnel behavior description text; the equipment reading text and / or personnel behavior description text and the category label are determined as the feature text of the current inspection image; the feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into the pre-trained large language model to obtain the image description information of the current inspection image output by the large language model, and the image description information is provided to the target user. The image description information is used to describe whether the current inspection image contains abnormal key power equipment, abnormal personnel and abnormal objects, which solves the inspection problem in the related technology. The inspection image processing scheme requires manual operation to process the images taken by the drone during the inspection of the power line. It is impossible to automatically process the images taken by the drone during the inspection of the power line to generate information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects. The labor cost and time cost are high, the processing efficiency is low, and the accuracy is difficult to guarantee. The images taken by the drone during the inspection of the power line can be automatically processed through the pre-trained target detection model, instance segmentation model and behavior recognition model to determine the feature text used to characterize the objects contained in the image. Then, based on the feature text of the image and the pre-trained large language model, the image description information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately obtained, and the image description information is provided to the target user, so that the images taken by the drone during the inspection of the power line are automatically processed during the inspection of the power line, and the information used to describe whether the image contains abnormal key power equipment, abnormal personnel and abnormal objects is accurately generated, which reduces the labor cost and time cost of the inspection image processing process and improves the processing efficiency and accuracy of the inspection image processing process.

[0107] In an optional implementation of an embodiment of the present invention, optionally, when the target detection module 301 performs the operation of performing target detection on the current inspection image through a pre-trained target detection model to determine the category label of the object in the current inspection image, it is specifically used to: input the current inspection image into the pre-trained target detection model, and obtain the category label of the object in the current inspection image output by the target detection model.

[0108] In an optional implementation of an embodiment of the present invention, optionally, the identification module 303 is specifically used for: if the category label includes critical power equipment and personnel, then cropping out the image area including the critical power equipment in the current inspection image to obtain a cropped image of the critical power equipment, and inputting the cropped image of the critical power equipment into a pre-trained instance segmentation model to obtain an equipment reading text of the cropped image of the critical power equipment output by the instance segmentation model; cropping out the image area including personnel in the current inspection image to obtain a cropped image of the personnel, and inputting the cropped image of the personnel into a pre-trained behavior recognition model to obtain a personnel behavior description text of the cropped image of the personnel output by the behavior recognition model.

[0109] In an optional implementation of the embodiment of the present invention, optionally, the feature text determination module 304 is specifically used to: determine the device reading text, the personnel behavior description text and the category label as the feature text of the current inspection image.

[0110] In an optional implementation of an embodiment of the present invention, optionally, the feature text determination module 304 is specifically used to: determine the device reading text, the personnel behavior description text, the category label and the image description information of the historical inspection image corresponding to the current inspection image as the feature text of the current inspection image.

[0111] In an optional implementation of an embodiment of the present invention, optionally, the description information determination module 305 is specifically used to: generate a feature text processing prompt instruction corresponding to the feature text of the current inspection image based on the feature text of the current inspection image and the feature text processing prompt instruction template; input the feature text processing prompt instruction into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and provide the image description information to a target user.

[0112] In an optional implementation manner of the embodiment of the present invention, optionally, the inspection image processing device also includes: an answer text determination module, which is used to input the question text processing prompt instruction corresponding to the inspection question text into the large language model after receiving the inspection question text sent by the target user, and obtain the answer text corresponding to the inspection question text output by the large language model; and an answer text sending module, which is used to send the answer text to the target user.

[0113] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0114] The inspection image processing device can execute the inspection image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the inspection image processing method.

[0115] Embodiment 4

[0116] Figure 4 The structure diagram of the electronic device 10 that can be used to implement the inspection image processing method of the embodiment of the present invention is shown. The components shown herein, their connections and relationships, and their functions are only for example, and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program built from the storage unit 18 to the random access memory (RAM) 13. In the random access memory (RAM) 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the read-only memory (ROM) 12 and the random access memory (RAM) 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0118] A number of components in the electronic device 10 are connected to an input / output (I / O) interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the inspection image processing method.

[0120] In some embodiments, the inspection image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via a read-only memory (ROM) 12 and / or a communication unit 19. When the computer program is built into a random access memory (RAM) 13 and executed by the processor 11, one or more steps of the inspection image processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the inspection image processing method in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The computer program for implementing the inspection image processing method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed completely on the machine, partially on the machine, partially on the machine and partially on a remote machine as an independent software package, or completely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A patrol inspection image processing method, characterized in that: include: After receiving the current inspection image of the target power line, performing target detection on the current inspection image by using a pre-trained target detection model to determine the category label of the object in the current inspection image; Determining whether the category tag includes critical power equipment and / or personnel; If the category label includes critical power equipment and / or personnel, the image area including the critical power equipment and / or personnel in the current inspection image is cropped out to obtain a cropped image of the critical power equipment and / or a cropped image of the personnel, and image recognition is performed on the cropped image of the critical power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text; Determine the device reading text and / or the personnel behavior description text and the category label as feature text of the current inspection image; The feature text processing prompt instruction corresponding to the feature text of the current inspection image is input into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and the image description information is provided to a target user; wherein the image description information is used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel and abnormal objects.

2. The inspection image processing method according to claim 1, characterized in that: Performing target detection on the current inspection image by using a pre-trained target detection model to determine the category label of the object in the current inspection image includes: The current inspection image is input into a pre-trained target detection model to obtain a category label of the object in the current inspection image output by the target detection model.

3. The inspection image processing method according to claim 1, characterized in that: If the category label includes key power equipment and / or personnel, the image area including the key power equipment and / or personnel in the current inspection image is cropped out to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and image recognition is performed on the cropped image of the key power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text, including: If the category label includes critical power equipment and personnel, the image region including the critical power equipment in the current inspection image is cropped out to obtain a cropped image of the critical power equipment, and the cropped image of the critical power equipment is input into a pre-trained instance segmentation model to obtain a device reading text of the cropped image of the critical power equipment output by the instance segmentation model; The image area containing personnel in the current inspection image is cropped out to obtain a personnel cropped image, and the personnel cropped image is input into a pre-trained behavior recognition model to obtain a personnel behavior description text of the personnel cropped image output by the behavior recognition model.

4. The inspection image processing method according to claim 3, characterized in that: Determining the device reading text and / or the personnel behavior description text and the category label as feature text of the current inspection image includes: The equipment reading text, the personnel behavior description text and the category label are determined as feature texts of the current inspection image.

5. The inspection image processing method according to claim 3, characterized in that: Determining the device reading text and / or the personnel behavior description text and the category label as feature text of the current inspection image includes: The device reading text, the personnel behavior description text, the category label, and image description information of the historical inspection image corresponding to the current inspection image are determined as feature text of the current inspection image.

6. The inspection image processing method according to claim 1, characterized in that: Inputting a feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, obtaining image description information of the current inspection image output by the large language model, and providing the image description information to a target user, including: Generate a feature text processing prompt instruction corresponding to the feature text of the current inspection image according to the feature text of the current inspection image and the feature text processing prompt instruction template; The feature text processing prompt instruction is input into a pre-trained large language model to obtain image description information of the current inspection image output by the large language model, and the image description information is provided to a target user.

7. The inspection image processing method according to claim 1, characterized in that: Also includes: After receiving the inspection question text sent by the target user, inputting the question text processing prompt instruction corresponding to the inspection question text into the large language model, and obtaining the answer text corresponding to the inspection question text output by the large language model; The answer text is sent to the target user.

8. A patrol inspection image processing device, characterized in that: include: A target detection module is used to, after receiving a current inspection image of a target power line, perform target detection on the current inspection image through a pre-trained target detection model to determine a category label of an object in the current inspection image; A tag processing module, used to determine whether the category tag contains key power equipment and / or personnel; a recognition module, configured to, if the category label includes key power equipment and / or personnel, crop the image region including the key power equipment and / or personnel in the current inspection image to obtain a cropped image of the key power equipment and / or a cropped image of the personnel, and perform image recognition on the cropped image of the key power equipment and / or the cropped image of the personnel through a pre-trained instance segmentation model and / or a behavior recognition model to determine the equipment reading text and / or the personnel behavior description text; A feature text determination module, used to determine the device reading text and / or the personnel behavior description text and the category label as the feature text of the current inspection image; A description information determination module is used to input the feature text processing prompt instruction corresponding to the feature text of the current inspection image into a pre-trained large language model, obtain the image description information of the current inspection image output by the large language model, and provide the image description information to a target user; wherein the image description information is information used to describe whether the current inspection image contains abnormal critical power equipment, abnormal personnel, and abnormal objects.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the inspection image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection image processing method according to any one of claims 1 to 7 when executed.

Images