Power grid equipment image recognition method and device, computer equipment and storage medium

Abstract

The invention relates to a power grid equipment image recognition method and device, computer equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: receiving target image data sent by shooting equipment through a pre-configured interface standard in an airborne image recognition system; the target image data is image data shot by a shooting device carrying an image sensor in a preset shooting scene; the image sensor communicates with the shooting equipment through an interface standard; determining a computing resource demand of the target image data, and if the computing resource demand is smaller than a computing resource of the edge computing terminal, identifying the target image data through a trained equipment identification model to obtain basic information of power grid equipment contained in the target image data; the equipment identification model is obtained by training based on an image training set containing classification labels of the power grid equipment; the basic information comprises category information and position information. By adopting the method, the image recognition efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, storage medium and computer program product for power grid equipment image recognition. Background Art

[0002] Intelligent image recognition technology uses computer vision technologies such as image processing and pattern recognition to analyze and interpret digital images, and to understand, describe and identify objects and their attributes in the image. This technology can not only improve the accuracy of medical diagnosis and improve traffic safety, but also play an important role in power grid equipment monitoring, drone inspection and other fields.

[0003] In the field of drone inspection and power grid equipment monitoring, the equipment in the airborne image recognition system currently on the market has problems such as single function, inflexible sensor access, and bulky computing terminal. As a result, the airborne image recognition system has insufficient recognition efficiency for images containing power grid equipment during image acquisition and image recognition. Summary of the invention

[0004] Based on this, it is necessary to provide a power grid equipment image recognition method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of power grid equipment recognition in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for image recognition of power grid equipment, which is applied to an edge computing terminal in an airborne image recognition system. The method comprises:

[0006] Receive target image data sent by a shooting device through an interface standard pre-configured in the airborne image recognition system; the target image data is image data shot by a shooting device equipped with an image sensor in a preset shooting scene; the image sensor communicates with the shooting device through the interface standard; the preset shooting scene includes at least one power grid device;

[0007] Determine the computing resource requirement of the target image data. If the computing resource requirement is less than the computing resource of the edge computing terminal, identify the target image data through a trained device recognition model to obtain basic information of the power grid equipment contained in the target image data; the device recognition model is trained based on an image training set containing classification labels of each power grid equipment; the basic information includes category information and location information.

[0008] In one embodiment, receiving the target image data sent by the shooting device through the interface standard pre-configured in the airborne image recognition system includes:

[0009] Generate a connection object between the edge computing terminal and the photographing device according to a pre-configured interface standard; the connection object is an instance of a connection between communication entities;

[0010] Broadcasting the connection object so that after the photographing device acquires the connection object, a communication connection is established with the edge computing terminal based on the connection object;

[0011] When the communication connection is established, target image data sent by the photographing device is received.

[0012] In one embodiment, the device recognition model includes an object recognition module and a location recognition module; the target image data is recognized by the trained device recognition model to obtain basic information of the power grid device contained in the target image data, including:

[0013] Input the target image data into the object recognition module, segment the target image data through the image segmentation unit included in the object recognition module to obtain a plurality of segmented sub-images, input each of the sub-images into the object classification unit included in the object recognition module to obtain category information of each power grid device included in the sub-image;

[0014] Each of the sub-images is input into the position identification module to obtain the position information of each power grid device in the sub-image; the position information is used to indicate the position of each power grid device in the target image data, and the distance value between each power grid device.

[0015] In one embodiment, after the target image data is identified by the trained device recognition model to obtain basic information of the power grid device contained in the target image data, the method further includes:

[0016] The basic information of the target image data and the target image data are sent to a cloud server, so that the cloud server can determine the abnormal analysis result of the target image data according to the number and distance value of each power grid device in the target image data for the target image data of the same shooting device at different times in the same preset shooting scene, and return the abnormal analysis result to the user terminal.

[0017] In one embodiment, sending the basic information of the target image data and the target image data to a cloud server includes:

[0018] Sending multiple target image data taken continuously at the same time by the same shooting device in the same preset shooting scene, and basic information of the multiple target image data to a cloud server, so that the cloud server can determine the abnormal analysis result of each power grid device according to the classification label of each power grid device in the continuous time, the first distance difference of the same power grid device in the continuous time, and the second distance difference of a pair of power grid devices in the continuous time;

[0019] Among them, when the classification label of the power grid device in a continuous time has a valid value at the previous time point and the classification label is missing at the next time point, it is determined that the power grid device is abnormal; when the first distance difference of the same power grid device in a continuous time is greater than a first preset value, it is determined that the same power grid device is abnormal; when the second distance difference of the pair of power grid devices in a continuous time is greater than a second preset value, it is determined that the pair of power grid devices is abnormal.

[0020] In one embodiment, the method further comprises:

[0021] The target image data is subjected to image preprocessing to obtain preprocessed target image data; the image preprocessing includes denoising, enhancement, cropping and shearing processing.

[0022] In one embodiment, the method further comprises:

[0023] Receiving parameter adjustment instructions and / or model training instructions sent by a user terminal;

[0024] According to the parameter adjustment instruction, updating at least one of the shooting parameters corresponding to the preset shooting scene, the device parameters of the shooting device, and the model parameters of the device recognition model;

[0025] The device identification model is trained according to the model training instruction to obtain a trained device identification model.

[0026] In a second aspect, the present application also provides a power grid equipment image recognition device. The device comprises:

[0027] A data receiving module is used to receive target image data sent by a shooting device through an interface standard pre-configured in an airborne image recognition system; the target image data is image data shot by a shooting device equipped with an image sensor in a preset shooting scene; the image sensor communicates with the shooting device through the interface standard; the preset shooting scene includes at least one power grid device;

[0028] An image recognition module is used to determine the computing resource requirements of the target image data. If the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is recognized through a trained device recognition model to obtain basic information of the power grid equipment contained in the target image data; the device recognition model is trained based on an image training set containing classification labels for each power grid equipment; the basic information includes category information and location information.

[0029] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0031] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0032] The above-mentioned power grid equipment image recognition method, device, computer equipment, storage medium and computer program product receive the target image data transmitted by the shooting device through the interface standard pre-configured in the airborne image recognition system. The shooting device uses the image sensor on board to shoot at least one power grid equipment in a preset shooting scene to obtain the target image data. The image sensor transmits the target image data to the shooting device through the interface standard, and transmits it from the shooting device to the edge computing terminal. The edge computing terminal determines the computing resource requirements of the target image data, and when the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is identified through the trained device recognition model to obtain the basic information of each power grid equipment in the target image data, and the basic information includes category information and location information. Since the target image data is transmitted in different devices by recording the interface standard pre-configured in the image recognition system, the transmission speed during image acquisition and image transmission can be improved. In addition, the target image data is acquired and recognized through the edge computing terminal. The communication distance between the edge computing terminal and the shooting device is short and the communication complexity is low, which can improve the efficiency of image acquisition and image recognition. The edge computing terminal does not need to send the target image data to the cloud server for recognition, thereby improving the image recognition efficiency of power grid equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] FIG1 is a diagram showing an application environment of a method for recognizing an image of a power grid device in one embodiment;

[0035] FIG2 is a schematic diagram of a flow chart of a method for recognizing an image of a power grid device in one embodiment;

[0036] FIG3 is a schematic flow chart of a step of receiving target image data in one embodiment;

[0037] FIG4 is a schematic diagram of a flow chart of a method for recognizing an image of a power grid device in another embodiment;

[0038] FIG5 is a schematic diagram of a flow chart of an airborne image recognition system in one embodiment;

[0039] FIG6 is a structural block diagram of a power grid equipment image recognition device in one embodiment;

[0040] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The power grid equipment image recognition method provided by the embodiment of the present application can be applied to the application environment shown in FIG. 1. Among them, the user terminal 102, the cloud server 104, the edge computing terminal 106 and the shooting device 108 in the figure are all components of the airborne image recognition system. The user terminal 102 sends a shooting instruction to the shooting device 108 through the network, so that the shooting device can shoot in a preset shooting scene to obtain the target image data. The shooting device 108 can send the target image data of real-time shooting or historical shooting to the user terminal 102 through the interface standard pre-configured in the airborne image recognition system, and the user terminal 102 can display the target image data on the display screen. The shooting device 108 can send the target image data to the edge computing terminal 106 through the interface standard, and the edge computing terminal 106 can determine the computing resource requirements of the target image data, and when the computing resources of the edge computing terminal meet the computing resource requirements, the target image data is identified through the trained device recognition model to obtain the basic information of each power grid device in the target image data. The edge computing terminal 106 can send the basic information and the target image data to the cloud server 104, and the cloud server can analyze the target image data. In addition, the user terminal 102 may send an analysis instruction to the cloud server 104 so that the cloud server analyzes the target image data.

[0043] The user terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart TVs, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The cloud server 104 may be implemented using an independent server or a server cluster consisting of multiple servers. The edge computing terminal 106 may be an edge gateway corresponding to the shooting device 108, a terminal equipped with an edge server, a smart camera, and a sensor, etc. The shooting device 108 may be a device carrying an image sensor, such as a drone, a camera, or a camera.

[0044] In an exemplary embodiment, as shown in FIG2 , a method for image recognition of power grid equipment is provided, which is described by taking the method applied to the edge computing terminal 106 in FIG1 as an example, and includes the following steps S202 to S204. Among them:

[0045] Step S202: receiving target image data sent by the shooting device through the interface standard pre-configured in the airborne image recognition system.

[0046] Among them, the airborne image recognition system is a system that includes various hardware devices and application software required for image capture, equipment monitoring, and image analysis of power grid equipment. The airborne image recognition system can monitor power grid equipment in the form of image data. Interface standards refer to the specifications and guidelines formulated when designing and building airborne image recognition to ensure the interoperability and compatibility between devices in different systems. Interface standards can define technical details such as the format of data exchange, communication protocols, electrical characteristics of interconnections, and physical forms. The target image data is image data captured by a shooting device equipped with an image sensor in a preset shooting scene. The target image data can be one image data or multiple time-continuous image data. The image sensor communicates with the shooting device through an interface standard. The preset shooting scene includes at least one power grid device. Power grid equipment can include power generation equipment, substation equipment, transmission equipment, distribution equipment, energy storage equipment, protection equipment, and automation equipment.

[0047] Specifically, the airborne image recognition system can configure corresponding interface standards for the edge computing terminal, the shooting device, and the image sensor during initialization. After the shooting device receives the shooting instruction, it can receive the target image data shot by the image sensor according to the shooting instruction. The shooting device can send the target image data to the edge computing terminal for subsequent image processing.

[0048] Step S204, determine the computing resource requirements of the target image data. If the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is identified through the trained device recognition model to obtain the basic information of the power grid equipment contained in the target image data.

[0049] Among them, the equipment recognition model is trained based on an image training set containing classification labels for each power grid equipment; the basic information includes category information and location information. The equipment recognition model can be obtained by combining one or more independent image recognition models, and each recognition model can be trained and used according to the required application scenario. The classification labels can include primary labels and secondary labels, and there is a hierarchical relationship between the primary labels and the secondary labels. For example, the primary labels of power grid equipment can be multiple labels such as power generation equipment, substation equipment, transmission equipment, and distribution equipment. The secondary labels corresponding to power generation equipment may include generators, photovoltaic modules, photovoltaic inverters, etc. In addition, the classification labels may also include other labels corresponding to equipment used for support and connection, such as cables and connectors.

[0050] Specifically, after receiving the target image data sent by the shooting device, the edge computing terminal can estimate the computing resource requirements of the target image data according to the size, resolution and quantity of the target image data, as well as the size and quantity of the target image data that may be generated later. In one example, the edge computing terminal can configure the initial resource value for image data of different sizes and resolutions, and determine the product of the initial resource value and the quantity as the total resource value. In addition, the edge computing terminal can obtain the shooting interval of the target image data, determine the quotient of the total resource value and the shooting interval, and thus obtain the computing resource requirements. The edge computing terminal can obtain the current available supply of its own computing resources. The edge computing terminal can compare the computing resource requirements of the target image data with the current available supply. If the computing resource requirements are less than the current available supply, the edge computing terminal can send the target image data to the device recognition model to obtain the recognition results of each power grid device, and integrate the recognition results to obtain the category information and location information of each power grid device contained in the target image data, and determine the category information and location information as basic information.

[0051] In the above-mentioned power grid equipment image recognition method, the target image data transmitted by the shooting device is received through the interface standard pre-configured in the airborne image recognition system. The shooting device uses the image sensor carried to shoot at least one power grid equipment in a preset shooting scene to obtain the target image data. The image sensor transmits the target image data to the shooting device through the interface standard, and transmits it from the shooting device to the edge computing terminal. The edge computing terminal determines the computing resource requirements of the target image data, and when the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is identified through the trained device recognition model to obtain the basic information of each power grid equipment in the target image data, and the basic information includes category information and location information. Since the target image data is transmitted in different devices by recording the interface standard pre-configured in the image recognition system, the transmission speed in the image acquisition and image transmission process can be improved. In addition, the target image data is a process of image acquisition and image recognition through the edge computing terminal. The communication distance between the edge computing terminal and the shooting device is short and the communication complexity is low, which can improve the efficiency of image acquisition and image recognition, and the edge computing terminal does not need to send the target image data to the cloud server for recognition, thereby improving the image recognition efficiency of the power grid equipment.

[0052] In an exemplary embodiment, as shown in FIG3 , the specific implementation process of the step of “receiving target image data sent by the shooting device through the interface standard pre-configured in the airborne image recognition system” includes steps S302 to S306. Among them:

[0053] Step S302: Generate a connection object between the edge computing terminal and the shooting device according to a pre-configured interface standard.

[0054] Specifically, a connection object is an instance of a connection between communication entities. The edge computing terminal can generate a connection object corresponding to at least one camera corresponding to the edge computing terminal according to the interface standard pre-configured in the airborne image recognition system. One connection object corresponds to one camera, thereby avoiding the sharing of one connection object by the camera devices, resulting in insufficient security and stability of data transmission.

[0055] Step S304: broadcast the connection object so that after the photographing device obtains the connection object, a communication connection with the edge computing terminal is established based on the connection object.

[0056] Specifically, the edge computing terminal can broadcast the generated multiple connection objects in real time. Each connection object has an identifier of a corresponding shooting device. If the identifier of the shooting device corresponds to the identifier stored in the connection object, the shooting device can consume the connection object, thereby establishing a communication connection between the connection object and the edge computing terminal. An edge computing terminal can be connected to at least one shooting device for communication.

[0057] Step S306: When the communication connection is established, receiving the target image data sent by the shooting device.

[0058] Specifically, when a communication connection is established with any shooting device, the edge computing terminal can receive the target image data sent by the shooting device in real time and update the target image data in real time over time.

[0059] In this embodiment, by creating a connection object through an interface standard, each shooting device can independently communicate with the edge computing terminal and receive target image data through the communication connection, which can improve the efficiency and security of transmitting the target image data.

[0060] In an exemplary embodiment, the device recognition model includes an object recognition module and a location recognition module. The specific implementation process of the step of "recognizing the target image data through the trained device recognition model to obtain the basic information of the power grid device contained in the target image data" includes:

[0061] The target image data is input into the object recognition module, and the target image data is segmented by the image segmentation unit included in the object recognition module to obtain multiple segmented sub-images, and each sub-image is input into the object classification unit included in the object recognition module to obtain the category information of each power grid device included in the sub-image. Each sub-image is input into the position recognition module to obtain the position information of each power grid device in the sub-image.

[0062] The location information is used to indicate the location of each power grid device in the target image data, as well as the distance value between each power grid device. The device recognition model includes an object recognition module and a location recognition module. The object recognition module is used to recognize each object contained in the image data, and classify each object to determine whether the object is a power grid device, and what type of power grid device it is. The location recognition module is used to determine the location information of the detection frame of each object in the image data, for example, the coordinate value in the coordinate axis of the image data.

[0063] Specifically, the edge computing terminal inputs the target image data into the object recognition module of the device recognition model. The object recognition module can be composed of multiple independent image recognition models, that is, it can be composed of multiple independent object classification units. Each image recognition model can classify different primary tags or secondary tags. For example, there may be a first image recognition model for identifying various cables, a second image recognition model for identifying various inverters, and so on. The object recognition module can mark each object in the target image data by the contour box generated by the image segmentation unit, and divide the target image data with the contour box as the dividing line to obtain multiple sub-images. For each sub-image, the edge computing terminal can determine the category information of the power grid equipment contained in each sub-image through different image recognition models.

[0064] Afterwards, the edge computing terminal can input each sub-image into the position recognition module. According to the coordinate system of the target image data and the position of the sub-image in the target image data, the coordinate value of each power grid device in the sub-image in the target image data can be determined. And the coordinate values ​​of two power grid devices are calculated to determine the distance value between the two power grid devices, thereby obtaining the distance value between any two power grid devices. Finally, the edge computing terminal can obtain the target image data containing the category information of each power grid device, as well as the location information of each power grid device.

[0065] In this embodiment, the image segmentation unit can segment each object in the target image data to obtain multiple sub-images, and the object classification unit can determine the category information of the power grid equipment contained in the sub-image. Based on this, the position information of each power grid equipment in the target image can also be determined by the position recognition module, which can improve the accuracy of classification and position recognition.

[0066] In an exemplary embodiment, after the step of "identifying the target image data by using the trained device recognition model to obtain basic information of the power grid device contained in the target image data", the method further includes:

[0067] The basic information and target image data of the target image data are sent to the cloud server, so that the cloud server can determine the abnormal analysis results of the target image data according to the number and distance value of each power grid device in the target image data for the target image data of the same shooting device at different times in the same preset shooting scene, and return the abnormal analysis results to the user terminal.

[0068] Specifically, the edge computing terminal can send the basic information of the target image data to the cloud server through the interface standard. The cloud server can store the target image data. The cloud server can analyze the target image data after receiving the analysis instruction sent by the user terminal. With respect to the preset shooting scene and shooting device in the analysis instruction, the cloud server can use the target image data shot by the same shooting device in the same preset shooting scene as a group of data to be analyzed. Usually, the data to be analyzed is composed of multiple image data at different times. The cloud server can determine the number of power grid devices contained in each image data in the data to be analyzed, and the distance value between any two power grid devices. If the number of power grid devices is less than the preset number, and there is a distance value between two power grid devices that is less than the preset value, it can be determined that the abnormal analysis result of the target image data is abnormal. The analysis strategy for determining the abnormal analysis result can be configured according to actual needs.

[0069] In this embodiment, the target image data is analyzed by the cloud server, which can store target image data acquired by multiple shooting devices and multiple preset shooting scenes over a long period of time, and the analysis process needs to acquire a certain amount of data to ensure the quality of the analysis. In addition, the computing resources of the cloud server are much greater than those of the edge computing terminal, which can reduce the computing burden of the edge computing terminal and improve the accuracy and efficiency of the analysis.

[0070] In an exemplary embodiment, the specific implementation process of the step of "sending the basic information of the target image data and the target image data to the cloud server" includes:

[0071] Multiple target image data taken continuously in the same preset shooting scene by the same shooting device and the basic information of the multiple target image data are sent to the cloud server, so that the cloud server can determine the abnormal analysis results of each power grid device according to the classification label of each power grid device in the continuous time, the first distance difference of the same power grid device in the continuous time, and the second distance difference of a pair of power grid devices in the continuous time.

[0072] Among them, when the classification labels of the power grid equipment in a continuous time have a valid value at the previous time point and the classification labels are missing at the next time point, it is determined that there is an abnormality in the power grid equipment; when the first distance difference of the same power grid equipment in a continuous time is greater than a first preset value, it is determined that there is an abnormality in the same power grid equipment; when the second distance difference of a pair of power grid equipment in a continuous time is greater than a second preset value, it is determined that there is an abnormality in a pair of power grid equipment.

[0073] Specifically, the edge computing terminal can send multiple time-continuous target image data taken by the same shooting device in the same preset scene, as well as the basic information corresponding to the target image data, to the cloud server. The cloud server can determine each power grid device contained in the target image data, as well as the location information of the power grid device based on the basic information of each target image data. For different target image data, they can be sorted in chronological order, and the classification label and coordinate value of the same power grid device in each target image data can be determined. Determine the classification label of the power grid device with the same coordinate value of two time-continuous target image data. If the classification labels are the same, it is determined that the power grid device has not changed; if the classification labels are different, it is determined that the power grid device has changed, that is, there is an abnormality.

[0074] The cloud server can determine the first distance difference between the same power grid device with the same classification label in two time-continuous target image data. For example, the cloud server can determine the first coordinate of the first device in the first image, determine the second coordinate of the first device in the second image, and determine the distance between the first coordinate and the second coordinate as the first distance difference of the power grid device in different image data. If the first distance difference is greater than the first preset value, it is determined that the power grid device is abnormal.

[0075] The cloud server can determine the second distance difference between any two power grid devices in two time-continuous target image data. For example, the first image and the second image are continuous in time. The cloud server can determine the first coordinate of the first device in the first image, the second coordinate of the second device, and determine the first difference between the two devices. The cloud server can determine the third coordinate of the first device in the second image, determine the fourth coordinate of the second device in the second image, and determine the second difference between the two devices. The difference between the first difference and the second difference is determined to be the second distance difference. If the second distance difference is greater than the second preset value, it can be determined that there is an abnormality between the first device and the second device. Among them, the preset value and the judgment strategy can be configured by the user terminal.

[0076] In this embodiment, the cloud server analyzes multiple target image data taken continuously at the same time by the same shooting device in the same preset shooting scene. The cloud server can determine whether there are abnormalities in each power grid device based on the classification label, the first distance difference and the second distance difference of the power grid device at the continuous time, which can improve the accuracy and completeness of the abnormal analysis results.

[0077] In an exemplary embodiment, the method further includes: performing image preprocessing on the target image data to obtain preprocessed target image data; the image preprocessing includes denoising, enhancement, cropping and shearing processing.

[0078] Specifically, the edge computing terminal can rotate and crop each target image data to achieve the image parameters pre-configured in the preset shooting scene. The resolution and size of the target image data meet the image parameters. In addition, the edge computing terminal can perform denoising and image enhancement processing on each target image data to obtain the pre-processed target image data.

[0079] In this embodiment, by performing image preprocessing on the target image data, the format standard of the target image data can be unified, and the uniformity and image quality of the image data can be improved, so that the recognition accuracy, analysis accuracy, recognition efficiency and analysis efficiency can be improved during subsequent recognition and analysis.

[0080] In an exemplary embodiment, the method also includes: receiving parameter adjustment instructions and / or model training instructions sent by a user terminal; updating at least one of the shooting parameters corresponding to a preset shooting scene, the device parameters of the shooting device, and the model parameters of the device recognition model according to the parameter adjustment instructions; and training the device recognition model according to the model training instructions to obtain a trained device recognition model.

[0081] Specifically, the edge computing terminal can receive parameter adjustment instructions sent by the user. The parameter adjustment instruction may include shooting parameters corresponding to the preset shooting scene. For example, shooting parameters may include shooting angles, shooting heights, shooting times, and other parameters used to control the shooting position and shooting time of the shooting device. The parameter adjustment instruction may include device parameters of the shooting device. For example, the device parameters include the device's shooting mode, captured image resolution, aperture, and other internal parameters of the device. The parameter adjustment instruction may include training parameters, hyperparameters, and other parameters internal to the device recognition model in the edge computing terminal. Based on this, the edge computing terminal can identify the parameter type in the parameter adjustment instruction, determine to send the parameter adjustment instruction to the corresponding device recognition model or shooting device, and update the corresponding parameters.

[0082] If the edge computing terminal receives a model training instruction sent by a user terminal, it can perform training according to at least one device recognition model corresponding to the model training instruction, thereby improving the recognition accuracy of the device recognition model.

[0083] In this embodiment, the shooting device and the image recognition model can be iteratively optimized through parameter adjustment instructions and model training instructions, thereby improving the recognition accuracy.

[0084] The specific implementation process of the above-mentioned power grid equipment image recognition method is described in detail below in conjunction with a specific embodiment. The embodiment of the present application aims to achieve efficient and accurate recognition of target objects through a comprehensive airborne image recognition system, and is applied to multiple scenarios such as drone inspections and power grid equipment monitoring. The embodiment of the present application divides the data processing flow into multiple modules through modular design, and each module performs different functions to achieve data generation, processing, analysis and final output. The airborne image recognition system includes: a data generation module, a data preprocessing module, an image recognition module, a data analysis module and a user interaction module.

[0085] Specifically, the executor of the data generation module is the image sensor carried by the drone. The function of the data generation module is: during the flight of the drone, the drone captures the image data of the target object through the image sensor carried (such as a visible light camera, an infrared camera, etc.). The data flow of the data generation module is that the image data is sent to the ground station or the edge computing terminal through wireless transmission.

[0086] For the data preprocessing module, its execution subject is the edge computing terminal. The data preprocessing module can preprocess the received image data, including denoising, enhancement, cropping, etc., to improve the image quality and reduce the difficulty of subsequent processing. The preprocessed image data can be passed from the data preprocessing module to the image recognition module.

[0087] For the image recognition module, its execution subject is the edge computing terminal or the cloud server (determined by the computing resource requirements. If the computing resources are greater than the available resources of the edge computing terminal, it is determined to use the cloud server for execution. If the computing resources are less than the available resources of the edge computing terminal, it is determined to use the edge computing terminal for execution). The image recognition module can use deep learning algorithms (such as convolutional neural networks) to identify preprocessed image data and output information such as the category and location of the target object. Finally, the image recognition module can pass the recognition results to the data analysis module.

[0088] For the data analysis module, its execution body is the cloud server. The data analysis module can analyze the recognition results, including the number, distribution, status, etc. of the target objects, and generate reports or warning information. The data analysis module can transmit the analysis results to the user terminal or the designated receiving system.

[0089] For the user interaction module, its execution subject is the user terminal (such as mobile phone, computer, etc.). The user interaction module can provide a user-friendly interface, display recognition results and analysis reports, and allow users to perform operations such as parameter adjustment and model training. The user terminal can initiate user instructions and parameter settings through the user interaction module, and iterative optimization is performed by passing user instructions and parameter settings to the data preprocessing module and the image recognition module.

[0090] In one example, as shown in FIG4 , the user terminal is a drone controller, and the drone controller can send control instructions to the drone and receive the video stream returned by the drone. The drone controller can transmit data through the 4G network RTK signal. The shooting device (drone) can determine the flight route and whether to shoot images according to the control instructions. For example, the control instruction can determine that the drone's image sensor automatically centers and shoots, and automatically zooms after locking the target. If the backlight is overexposed, the shooting parameters can be automatically adjusted, and finally the number of video frames shot is determined to be 25 frames per second. After the drone shoots an image, the video stream can be transmitted to the edge computing terminal first, and the objects in the image can be identified by the edge computing terminal to obtain an image containing the labels of each object, and returned to the user terminal so that the user terminal can display the image containing the device label.

[0091] In one example, as shown in FIG5 , a drone carries an image sensor to capture image data during flight. The image data is sent to the edge computing terminal via wireless transmission. The edge computing terminal preprocesses the image data. The preprocessed image data is passed to the image recognition module for recognition. The recognition result is passed to the data analysis module for analysis. The analysis result is displayed to the user through the user interaction module. The user adjusts parameters or trains the model according to needs, and passes instructions to the data preprocessing module and the image recognition module through the user interaction module for iterative optimization.

[0092] Specifically, the data generation module inputs: image data during the flight of the drone. Output: original image data. Relationship with other modules: passes the original image data to the data preprocessing module. The data preprocessing module inputs: original image data. Output: preprocessed image data. Relationship with other modules: receives the output of the data generation module and passes the preprocessed image data to the image recognition module.

[0093] Image recognition module, input: preprocessed image data. Output: category, location and other information of the target object. Relationship with other modules: receives the output of the data preprocessing module and passes the recognition results to the data analysis module.

[0094] Data analysis module, input: recognition results. Output: analysis report or warning information. Relationship with other modules: receives the output of the image recognition module and passes the analysis results to the user interaction module.

[0095] User interaction module, input: user instructions and parameter settings. Output: display recognition results and analysis reports. Relationship with other modules: receive user instructions and parameter settings, pass them to the data preprocessing module and image recognition module for iterative optimization; display the output results of the data analysis module.

[0096] In this example, through the above modular design, the present invention realizes the comprehensive application of airborne image intelligent recognition technology, improves the scalability, practicality and convenience of the system, and provides strong technical support for the fields of drone inspection, power grid equipment monitoring, etc.

[0097] In one example, this method can be applied to the fields of drone inspection and power grid equipment monitoring. For example, in power inspection, drones are equipped with image sensors to photograph power grid equipment, and the system identifies and analyzes the images, and timely warns of abnormal equipment, thereby improving inspection efficiency. When monitoring power grid equipment, the system can monitor power grid equipment in real time, identify equipment status, fault type, etc., and provide a decision-making basis for power grid operation and maintenance. The system analyzes historical data of power grid equipment, predicts equipment life, fault trends, etc., and provides a scientific basis for power grid planning.

[0098] In terms of data processing efficiency, the embodiment of the present application divides the complex image recognition process into multiple independent functional modules through modular design, realizes high-speed data transmission and processing, and significantly improves the response speed and recognition accuracy of the system. Especially in application scenarios such as drone inspection and power grid equipment monitoring, the system can capture the image data of the target object in real time, and quickly pre-process, identify and analyze it, providing timely and accurate information support for operation and maintenance personnel.

[0099] In terms of recognition accuracy, this study uses advanced deep learning algorithms, and through a large amount of training data and optimized model structure, it improves the system's recognition accuracy and robustness for target objects. Even in complex environments and conditions, such as lighting changes, occlusion, angle changes, etc., the system can still maintain high recognition performance, providing reliable technical support for various application scenarios.

[0100] The technology also has good scalability and flexibility. The system can flexibly adjust sensor configuration, algorithm parameters and recognition models according to different application scenarios and requirements to adapt to various complex environments and task requirements. At the same time, through the design of the user interaction module, users can easily view recognition results and analysis reports, adjust parameters and train models, and achieve continuous optimization and improvement of the system.

[0101] Some technical terms are as follows:

[0102] 1. Airborne image intelligent recognition technology: This is a technology that combines image recognition technology with flying platforms such as drones. By capturing images of target objects during flight and using algorithms to intelligently analyze the images, it can achieve rapid and accurate recognition of target objects.

[0103] 2. Deep Learning Algorithms: A machine learning method that builds a deep neural network model to learn from a large amount of data, so that it can automatically extract features from the data and perform tasks such as classification and recognition. In the field of image recognition, deep learning algorithms have excellent performance.

[0104] 3. UAV inspection: A technology that uses drones equipped with sensors (such as cameras) to inspect and monitor specific areas or targets. It is widely used in power inspection, agricultural monitoring and other fields.

[0105] 4. Grid equipment monitoring: Real-time monitoring and data analysis of various equipment in the power grid (such as transformers, lines, etc.) to detect equipment failures, predict equipment life, etc., and provide technical support for the safe operation of the power grid.

[0106] 5. Image sensor: A device that can convert optical images into electrical signals, widely used in digital cameras, webcams and other devices. In airborne image intelligent recognition technology, image sensors are used to capture image data of target objects.

[0107] 6. Edge computing terminal: A computing device located at the edge of the network that can process and analyze data from IoT devices, reduce data transmission delays, and improve system response speed. In airborne image intelligent recognition technology, edge computing terminals can pre-process and preliminarily recognize image data.

[0108] 7. User interaction module: A module that allows users to interact with computer systems, which can receive user input instructions and display system output results. In airborne image intelligent recognition technology, the user interaction module can provide users with a friendly interface, display recognition results and analysis reports, and allow users to perform operations such as parameter adjustment and model training.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a power grid equipment image recognition device for implementing the power grid equipment image recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more power grid equipment image recognition device embodiments provided below can refer to the limitations of the power grid equipment image recognition method above, and will not be repeated here.

[0111] In an exemplary embodiment, as shown in FIG6 , a power grid equipment image recognition device 600 is provided, comprising: a data receiving module 601 and an image recognition module 602, wherein:

[0112] The data receiving module 601 is used to receive target image data sent by the shooting device through the interface standard pre-configured in the airborne image recognition system; the target image data is image data shot by the shooting device equipped with an image sensor in a preset shooting scene; the image sensor and the shooting device communicate through the interface standard; the preset shooting scene includes at least one power grid device;

[0113] The image recognition module 602 is used to determine the computing resource requirements of the target image data. If the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is recognized through a trained device recognition model to obtain the basic information of the power grid equipment contained in the target image data; the device recognition model is trained based on an image training set containing classification labels for each power grid device; the basic information includes category information and location information.

[0114] Furthermore, the data receiving module 601 is specifically used to: generate a connection object between the edge computing terminal and the shooting device according to a pre-configured interface standard; the connection object is an instance of a connection between communication entities; broadcast the connection object so that after the shooting device obtains the connection object, it establishes a communication connection with the edge computing terminal based on the connection object; when the communication connection is established, receive the target image data sent by the shooting device.

[0115] Furthermore, the equipment recognition model includes an object recognition module and a position recognition module, and the image recognition module 602 is specifically used to: input the target image data into the object recognition module, segment the target image data through the image segmentation unit included in the object recognition module, and obtain multiple sub-images after segmentation, and input each sub-image into the object classification unit included in the object recognition module to obtain the category information of each power grid device contained in the sub-image; input each sub-image into the position recognition module to obtain the position information of each power grid device in the sub-image; the position information is used to indicate the position of each power grid device in the target image data, as well as the distance value between each power grid device.

[0116] Furthermore, the device also includes an identification result sending module, which is used to: send the basic information of the target image data and the target image data to the cloud server, so that the cloud server can determine the abnormal analysis results of the target image data according to the number and distance value of each power grid device in the target image data for the target image data of the same shooting device at different times in the same preset shooting scene, and return the abnormal analysis results to the user terminal.

[0117] Furthermore, the recognition result sending module is specifically used to: send multiple target image data taken continuously at the same time by the same shooting device in the same preset shooting scene, and basic information of the multiple target image data to the cloud server, so that the cloud server can determine the abnormal analysis results of each power grid device according to the classification label of each power grid device in the continuous time, the first distance difference of the same power grid device in the continuous time, and the second distance difference of a pair of power grid devices in the continuous time; wherein, among the classification labels of the power grid devices in the continuous time, if there is a valid value for the classification label at the previous time point and the classification label is missing at the next time point, it is determined that the power grid device is abnormal; if the first distance difference of the same power grid device in the continuous time is greater than the first preset value, it is determined that the same power grid device is abnormal; if the second distance difference of a pair of power grid devices in the continuous time is greater than the second preset value, it is determined that the pair of power grid devices is abnormal.

[0118] Furthermore, the device also includes a preprocessing module, which is used to: perform image preprocessing on the target image data to obtain preprocessed target image data; the image preprocessing includes denoising, enhancement, cropping and shearing processing.

[0119] Furthermore, the device also includes an instruction execution module, which is used to: receive parameter adjustment instructions and / or model training instructions sent by the user terminal; update at least one of the shooting parameters corresponding to the preset shooting scene, the device parameters of the shooting device, and the model parameters of the device recognition model according to the parameter adjustment instructions; and train the device recognition model according to the model training instructions to obtain a trained device recognition model.

[0120] Each module in the above-mentioned power grid equipment image recognition device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0121] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target image data captured by the shooting device. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for image recognition of power grid equipment is implemented.

[0122] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0123] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0125] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for recognizing an image of a power grid device, characterized in that: An edge computing terminal applied to an airborne image recognition system, the method comprising: Receive target image data sent by a shooting device through an interface standard pre-configured in the airborne image recognition system; the target image data is image data shot by a shooting device equipped with an image sensor in a preset shooting scene; the image sensor communicates with the shooting device through the interface standard; the preset shooting scene includes at least one power grid device; Determine the computing resource requirement of the target image data. If the computing resource requirement is less than the computing resource of the edge computing terminal, identify the target image data through a trained device recognition model to obtain basic information of the power grid equipment contained in the target image data; the device recognition model is trained based on an image training set containing classification labels of each power grid equipment; the basic information includes category information and location information.

2. The method according to claim 1, characterized in that The receiving target image data sent by the shooting device through the interface standard pre-configured in the airborne image recognition system includes: Generate a connection object between the edge computing terminal and the photographing device according to a pre-configured interface standard; the connection object is an instance of a connection between communication entities; Broadcasting the connection object so that after the photographing device acquires the connection object, a communication connection is established with the edge computing terminal based on the connection object; When the communication connection is established, target image data sent by the photographing device is received.

3. The method according to claim 1, characterized in that: The device recognition model includes an object recognition module and a location recognition module; the target image data is recognized by the trained device recognition model to obtain basic information of the power grid equipment contained in the target image data, including: Input the target image data into the object recognition module, segment the target image data through the image segmentation unit included in the object recognition module to obtain a plurality of segmented sub-images, input each of the sub-images into the object classification unit included in the object recognition module to obtain category information of each power grid device included in the sub-image; Each of the sub-images is input into the position identification module to obtain the position information of each power grid device in the sub-image; the position information is used to indicate the position of each power grid device in the target image data, and the distance value between each power grid device.

4. The method according to claim 1, characterized in that: After the target image data is identified by the trained device identification model to obtain basic information of the power grid device contained in the target image data, the method further includes: The basic information of the target image data and the target image data are sent to a cloud server, so that the cloud server can determine the abnormal analysis result of the target image data according to the number and distance value of each power grid device in the target image data for the target image data of the same shooting device at different times in the same preset shooting scene, and return the abnormal analysis result to the user terminal.

5. The method according to claim 4, characterized in that The sending the basic information of the target image data and the target image data to the cloud server includes: Sending multiple target image data taken continuously at the same time by the same shooting device in the same preset shooting scene, and basic information of the multiple target image data to a cloud server, so that the cloud server can determine the abnormal analysis result of each power grid device according to the classification label of each power grid device in the continuous time, the first distance difference of the same power grid device in the continuous time, and the second distance difference of a pair of power grid devices in the continuous time; Among them, when the classification label of the power grid device in a continuous time has a valid value at the previous time point and the classification label is missing at the next time point, it is determined that the power grid device is abnormal; when the first distance difference of the same power grid device in a continuous time is greater than a first preset value, it is determined that the same power grid device is abnormal; when the second distance difference of the pair of power grid devices in a continuous time is greater than a second preset value, it is determined that the pair of power grid devices is abnormal.

6. The method according to claim 1, characterized in that The method further comprises: The target image data is subjected to image preprocessing to obtain preprocessed target image data; the image preprocessing includes denoising, enhancement, cropping and shearing processing.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Receiving parameter adjustment instructions and / or model training instructions sent by a user terminal; According to the parameter adjustment instruction, updating at least one of the shooting parameters corresponding to the preset shooting scene, the device parameters of the shooting device, and the model parameters of the device recognition model; The device identification model is trained according to the model training instruction to obtain a trained device identification model.

8. A power grid equipment image recognition device, characterized in that: The device comprises: A data receiving module is used to receive target image data sent by a shooting device through an interface standard pre-configured in an airborne image recognition system; the target image data is image data shot by a shooting device equipped with an image sensor in a preset shooting scene; the image sensor communicates with the shooting device through the interface standard; the preset shooting scene includes at least one power grid device; An image recognition module is used to determine the computing resource requirements of the target image data. If the computing resource requirements are less than the computing resources of the edge computing terminal, the target image data is recognized through a trained device recognition model to obtain basic information of the power grid equipment contained in the target image data; the device recognition model is trained based on an image training set containing classification labels for each power grid equipment; the basic information includes category information and location information.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images