Data processing method and device for unmanned aerial vehicle inspection

Abstract

The invention discloses a data processing method and device for unmanned aerial vehicle inspection. The method comprises the following steps: controlling an unmanned aerial vehicle to obtain inspection image information; performing image segmentation processing on the inspection image information by using the first processing terminal to obtain target processing image information; and determining target control signal information based on the second processing terminal and the target processing image information.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a data processing method and device for unmanned aerial vehicle inspection. Background Art

[0002] Currently, when dealing with power system failures, manual on-site processing is still required. This will reduce work efficiency, affect power quality, and even prolong power outages. Compared with traditional line inspections, drone line inspections have the advantages of cost-effectiveness, high efficiency, high safety, accurate data collection, strong flexibility, and environmental protection and energy saving. Drone line inspections can effectively improve the stability of the power system and gradually become the main trend of current power system line inspections. Therefore, a data processing method and device for drone inspections are provided to improve the control efficiency of drone inspections and the accuracy of line inspection control, so as to achieve the use of drone line inspections under the premise of a lighter workload and reduce the manpower cost of power system line inspections. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a data processing method and device for drone inspection, which is conducive to improving the control efficiency and line inspection control accuracy of drone inspection, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses a data processing method for drone inspection, the method comprising:

[0005] Control the drone to obtain inspection image information;

[0006] Using the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information;

[0007] Based on the second processing terminal and the target processing image information, target control signal information is determined.

[0008] A second aspect of an embodiment of the present invention discloses a data processing device for drone inspection, the device comprising:

[0009] The acquisition module is used to control the UAV to obtain inspection image information;

[0010] A processing module, used for performing image segmentation processing on the inspection image information by using the first processing terminal to obtain target processed image information;

[0011] The determination module is used to determine the target control signal information based on the second processing terminal and the target processing image information.

[0012] The third aspect of the present invention discloses another data processing device for unmanned aerial vehicle inspection, the device comprising:

[0013] A memory storing executable program code;

[0014] a processor coupled to the memory;

[0015] The processor calls the executable program code stored in the memory to execute some or all of the steps in the data processing method for drone inspection disclosed in the first aspect of the embodiment of the present invention.

[0016] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the data processing method for drone inspection disclosed in the first aspect of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a schematic diagram of a scenario of a data processing system for drone inspection provided by an embodiment of the present invention;

[0019] Figure 2 It is a flow chart of a data processing method for unmanned aerial vehicle inspection disclosed in an embodiment of the present invention;

[0020] Figure 3 It is a structural schematic diagram of a data processing device for unmanned aerial vehicle inspection disclosed in an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of another data processing device for unmanned aerial vehicle inspection disclosed in an embodiment of the present invention;

[0022] Figure 5 is a structural schematic diagram of a target image processing model disclosed in an embodiment of the present invention;

[0023] Figure 6 is a structural schematic diagram of a first multidimensional feature extraction module disclosed in an embodiment of the present invention;

[0024] Figure 7 is a structural schematic diagram of a first deep feature extraction module disclosed in an embodiment of the present invention;

[0025] Figure 8 It is a structural schematic diagram of a brain-computer processing model disclosed in an embodiment of the present invention;

[0026] Fig. 9 It is a schematic diagram of a scenario of target processing image information provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0029] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0031] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0032] The embodiments of the present application provide a data processing method, apparatus, computer equipment, and computer-readable storage medium for drone inspection, which are described in detail below.

[0033] See also Figure 1 , Figure 1 Schematic diagram of a scenario of a data processing system for drone inspection provided in an embodiment of the present application. The data processing system for drone inspection may include a computer device 100, in which a data processing device for drone inspection is integrated, such as Figure 1 Computer equipment in.

[0034] In the embodiment of the present application, the computer device 100 is mainly used to control the drone to obtain inspection image information;

[0035] Using the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information;

[0036] Based on the second processing terminal and the target processing image information, target control signal information is determined.

[0037] It can improve the control efficiency of drone inspections and the accuracy of line inspection control, enable the use of drones for line inspections with a lighter workload, and reduce the manpower cost of power system line inspections.

[0038] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0039] It is understandable that the computer device 100 used in the embodiments of the present application may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0040] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown. It can be understood that the data processing system for drone inspections may also include one or more other services, which are not specifically limited here.

[0041] In addition, if Figure 1 As shown, the data processing system for drone inspection may also include a memory 200 for storing data, such as image data, location information, etc.

[0042] It should be noted that Figure 1 The scenario diagram of the data processing system for drone inspection shown is merely an example. The data processing system and scenario for drone inspection described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the data processing system for drone inspection and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0043] The present invention discloses a data processing method and device for drone inspection, which is beneficial to improving the control efficiency and line inspection control accuracy of drone inspection, realizing the use of drones for line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection. The following are detailed descriptions.

[0044] Embodiment 1

[0045] See also Figure 2 , Figure 2 1 is a flow chart of a data processing method for drone inspection disclosed in an embodiment of the present invention. Figure 2The data processing method for drone inspection described above is applied to a management system, such as a local server or cloud server for management, and the embodiments of the present invention are not limited thereto. Figure 2 As shown, the data processing method for drone inspection may include the following operations:

[0046] 101. Control the drone to obtain inspection image information.

[0047] 102. Use the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information.

[0048] 103. Determine target control signal information based on the second processing terminal and the target processing image information.

[0049] It should be noted that the data processing method for drone inspection of the present application can realize semantic-level control of drones (starting line inspection, returning, etc.), and command-level control of drones (such as DJI drones) (de-icing, spraying fire, canceling execution, etc.). At the same time, the data processing method for drone inspection of the present application can realize automatic line inspection for the identification of transmission lines; realize automatic scanning and detection for the identification of insulators; and immediately convert obstacles such as ice cubes and kites into stimulation based on brain wave signals after identification (the stimulation generated by ice cubes corresponds to the instruction of swinging the de-icing stick, and the stimulation generated by kites corresponds to the instruction of spraying fire). The data processing method for drone inspection of the present application can minimize the manual burden while ensuring the efficiency of drone line inspection.

[0050] It should be noted that in the present application, when the second processing terminal displays the image, different object categories in the image are displayed by flashing at the same time. The flashing frequencies and phases of different image categories are shown in the following table:

[0051] Image Category Frequency / Hz Phase / ° Transmission Lines 0 0 insulator 0 0 Ice Cover 15 5 Kite 16 4 String 17 3

[0052] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0053] In an optional embodiment, the above-mentioned using the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information includes:

[0054] Receiving inspection image information sent by the drone at the first processing terminal to obtain received image information;

[0055] pre-processing the received image information at the first processing terminal to obtain first processed image information;

[0056] At the first processing terminal, the first processed image information is segmented using a target image processing model to obtain target processed image information.

[0057] It should be noted that, considering that the processor performance of the drone is relatively limited, the inspection images collected by the drone are sent to a first processing terminal equipped with a GPU to improve the processing efficiency and accuracy of the inspection images, which is not limited in the embodiment of the present invention.

[0058] It should be noted that the above preprocessing of the received image information is to convert the image into TIFF format and adjust the image size to 512×512, which is not limited in the embodiment of the present invention.

[0059] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0060] In another optional embodiment, Figure 5 As shown, the target image processing model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a thirteenth convolution module, a first normalization module, a second normalization module, a third normalization module, a fourth normalization module, a first upsampling module, a second upsampling module, a first fusion module, a first pooling module, a first multidimensional feature extraction module, a second multidimensional feature extraction module, a third multidimensional feature extraction module, a fourth multidimensional feature extraction module, a first deep feature extraction module, a second deep feature extraction module and a third deep feature extraction module; wherein,

[0061] The input end of the first convolution module is configured as the model input end of the target image processing model, the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the second normalization module; the output end of the second normalization module is connected to the input end of the first multidimensional feature extraction module; the output end of the first multidimensional feature extraction module is respectively connected to the input end of the first deep feature extraction module and the input end of the seventh convolution module; the output end of the first deep feature extraction module is connected to the input end of the second multidimensional feature extraction module; the output end of the second multidimensional feature extraction module is connected to the input end of the second deep feature extraction module; the output end of the second deep feature extraction module is connected to the input end of the third multidimensional feature extraction module; the output end of the third multidimensional feature extraction module is connected to the input end of the third deep feature extraction module; the output end of the third deep feature extraction module is connected to the input end of the fourth multidimensional feature extraction module; the output end of the fourth multidimensional feature extraction module is connected to the thirteenth convolution module The input end of the block; the output end of the thirteenth convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the first pooling module; the output end of the first pooling module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first upsampling module; the output end of the first upsampling module is connected to the input end of the first fusion module; the output end of the seventh convolution module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the fourth normalization module; the output end of the fourth normalization module is connected to the input end of the second activation module; the output end of the second activation module is connected to the input end of the eighth convolution module; the output end of the eighth convolution module is connected to the input end of the second upsampling module; the output end of the second upsampling module is configured as the model output end of the target image processing model.

[0062] It should be noted that the convolution kernels of the above-mentioned first convolution module, second convolution module, third convolution module, fourth convolution module, fifth convolution module and eighth convolution module are 3×3, which is not limited in the embodiment of the present invention.

[0063] It should be noted that the convolution kernels of the sixth convolution module, the seventh convolution module, and the thirteenth convolution module are 1×1, which is not limited in the embodiment of the present invention.

[0064] It should be noted that the above-mentioned first normalization module, second normalization module, third normalization module, and fourth normalization module are constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0065] It should be noted that the above-mentioned first pooling module is constructed based on the maximum pooling layer, which is not limited in the embodiment of the present invention.

[0066] It should be noted that the above-mentioned first fusion module is constructed based on the concat operation, which is not limited in the embodiment of the present invention.

[0067] It should be noted that the first upsampling module and the second upsampling module are constructed based on an upsampling operation (upsample), which is not limited in the embodiment of the present invention.

[0068] It should be noted that the target image processing model can reasonably utilize the correlation of feature channels to achieve in-depth and comprehensive capture of the global semantics and local semantics of the image, and improve the accuracy of image recognition and segmentation, which is not limited in the embodiments of the present invention. Furthermore, the training loss of the target image processing model can be a cross entropy loss function, a Focal loss function, etc., which is not limited in the embodiments of the present invention.

[0069] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0070] In yet another optional embodiment, Figure 6 As shown, the first multidimensional feature extraction module includes a classification network, a first pooling unit, a first connection unit, a second connection unit, a first activation unit, a first fusion unit, a second fusion unit, a third fusion unit, a fourth fusion unit, a first convolution unit, a second convolution unit, a third convolution unit, a feedforward neural network, a first normalization unit, a second normalization unit, a first dimension conversion unit, a second dimension conversion unit, and a multi-head attention unit; wherein,

[0071] The input end of the classification network is configured as the input end of the first multidimensional feature extraction module, and the output end of the classification network is respectively connected to the input end of the first pooling unit and the input end of the first fusion unit; the output end of the first pooling unit is connected to the input end of the first connection unit; the output end of the first connection unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the second connection unit; the output end of the second connection unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is respectively connected to the input end of the first convolution unit and the input end of the fourth fusion unit; the output end of the first convolution unit is connected to the input end of the first convolution unit; the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the first dimension conversion unit; the first dimension conversion The output end of the conversion unit is respectively connected to the input end of the first normalization unit and the input end of the second fusion unit; the output end of the first normalization unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is respectively connected to the input end of the second normalization unit and the input end of the third fusion unit; the output end of the second normalization unit is connected to the input end of the feedforward neural network; the output end of the feedforward neural network is connected to the input end of the third fusion unit; the output end of the third fusion unit is connected to the input end of the second dimension conversion unit; the output end of the second dimension conversion unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth fusion unit; the output end of the fourth fusion unit is configured as the output end of the first multidimensional feature extraction module.

[0072] It should be noted that the above classification network is constructed based on the VGG network model, which is not limited in the embodiment of the present invention.

[0073] It should be noted that the above-mentioned first pooling unit is constructed based on the global pooling layer, which is not limited in the embodiment of the present invention.

[0074] It should be noted that the first connection unit and the second connection unit are both constructed based on a one-dimensional fully connected layer, which is not limited in the embodiment of the present invention.

[0075] It should be noted that the above-mentioned first activation unit is constructed based on the RELE activation function, which is not limited in the embodiment of the present invention.

[0076] It should be noted that the first fusion unit is constructed based on a scale network, which is not limited in the embodiment of the present invention.

[0077] It should be noted that the convolution kernel size of the above-mentioned first convolution unit is 3×3, which is not limited in this embodiment of the present invention.

[0078] It should be noted that the second convolution unit and the third convolution unit are 1×1, which is not limited in this embodiment of the present invention.

[0079] It should be noted that the above-mentioned first dimension conversion unit is constructed based on a multi-dimensional to one-dimensional operation, that is, mapping multi-dimensional matrix features to non-overlapping one-dimensional vectors, which is not limited in the embodiment of the present invention.

[0080] It should be noted that the above-mentioned second dimension conversion unit is constructed based on a one-dimensional to multi-dimensional operation, that is, restoring multiple one-dimensional vectors to a multi-dimensional matrix of original features. It is an opposite feature operation to the first dimension conversion unit, and the embodiment of the present invention does not limit it.

[0081] It should be noted that the first normalization unit and the second normalization unit are constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0082] It should be noted that the above-mentioned multi-head attention unit is constructed based on the multi-head attention mechanism, which is not limited in the embodiments of the present invention.

[0083] It should be noted that the second fusion unit and the third fusion unit are constructed based on the add operation, which is not limited in the embodiment of the present invention.

[0084] It should be noted that the fourth fusion unit is constructed based on the concat operation, which is not limited in the embodiment of the present invention.

[0085] It should be noted that the above-mentioned first multidimensional feature extraction module processes features through convolution operations, nonlinear activation functions, etc., which not only reduces parameters and calculation amount, but also realizes local and global multi-dimensional feature extraction, thereby performing more comprehensive and in-depth feature extraction and analysis of the image, which is not limited in the embodiments of the present invention.

[0086] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0087] In yet another optional embodiment, Figure 7 As shown, the first deep feature extraction module includes a fourth convolution unit, a fifth convolution unit, a third normalization unit and a second activation unit; wherein,

[0088] The input end of the fourth convolution unit is configured as the input end of the first deep feature extraction module, the output end of the fourth convolution unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second activation unit; the output end of the second activation unit is configured as the output end of the first deep feature extraction module.

[0089] It should be noted that the convolution kernel sizes of the fourth convolution unit and the fifth convolution unit are 3×3, which is not limited in the embodiment of the present invention.

[0090] It should be noted that the third normalization unit is constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0091] It should be noted that the above-mentioned second activation unit is constructed based on the RELU activation function, which is not limited in the embodiment of the present invention.

[0092] It should be noted that after the first feature extraction module performs deep extraction processing on the feature information, the image size of the image features will be reduced by half, which is not limited in the embodiment of the present invention.

[0093] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0094] In an optional embodiment, the above-mentioned determining the target control signal information based on the second processing terminal and the target processing image information includes:

[0095] Displaying target processed image information at a second processing terminal;

[0096] During the signal acquisition period, the second processing terminal is used to acquire brain wave signal information; the brain wave signal information represents a brain wave signal of a continuous time period; the signal acquisition period represents the acquisition time period from when the second processing terminal starts to display the target processing image information to when the second processing terminal completes the acquisition of the brain wave signal;

[0097] The brain wave signal information is processed by using the brain-computer processing model at the second processing terminal to obtain the target control signal information.

[0098] It should be noted that the above-mentioned displaying the target processing image information on the second processing terminal means displaying the target processing image corresponding to the target processing image information on the display of the second processing terminal, which is not limited in the embodiment of the present invention. Fig. 9 As shown, the segmented and identified images of the insulator and the ice coating are displayed on the second processing terminal, which is not limited in this embodiment of the present invention.

[0099] It should be noted that the above-mentioned collection time period can be 1 second or 5 seconds, which is not limited in the embodiment of the present invention.

[0100] It should be noted that the above-mentioned signal acquisition cycle starts timing when the target processing image information is displayed, and the acquisition of brain wave signals (such as electroencephalography (EEG)) is terminated when the acquisition time period is reached, which is not limited in the embodiment of the present invention.

[0101] It should be noted that the above-mentioned brain wave signal is a signal generated when the user looks at the displayed target processing image, and the frequency of collecting brain wave signals can be 250Hz, which is not limited in the embodiment of the present invention. Furthermore, the above-mentioned acquisition device can be an EEG cap corresponding to the second processing terminal, which is not limited in the embodiment of the present invention. Furthermore, the above-mentioned second processing terminal also includes an amplifier and electrode wires that can be used in brain wave signal collection, which is not limited in the embodiment of the present invention.

[0102] It should be noted that the above-mentioned processing of brain wave signal information using the brain-computer processing model at the second processing terminal to obtain the target control signal information is that the host corresponding to the second processing terminal (such as the Android system or other operating system) first performs frequency decomposition (such as using FFT) on the brain wave signal information, and then uses the brain-computer processing model to perform signal classification processing on the matrix corresponding to the frequency information, and obtains the control classification representing the user's intended control information, and then generates the executable instruction code of the drone (i.e., the target control signal information) according to the control paradigm. Furthermore, the correspondence between the executable instruction code of the drone and the control classification can be shown in the following table:

[0103]

[0104] It should be noted that after the command code is determined, the command will be transmitted to the drone. After the ice is cleared, the drone will automatically hover and wait for the next command from the on-duty personnel, which is not limited in the embodiment of the present invention.

[0105] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0106] In another optional embodiment, Figure 8 As shown, the brain-computer processing model includes a ninth convolution module, a tenth convolution module, an eleventh convolution module, a twelfth convolution module, a fifth normalization module, a third activation module and a connection module; wherein,

[0107] The ninth convolution module is configured as the model input end of the brain-computer processing model, the output end of the ninth convolution module is connected to the input end of the tenth convolution module; the output end of the tenth convolution module is connected to the input end of the fifth normalization module; the output end of the fifth normalization module is connected to the input end of the eleventh convolution module; the output end of the eleventh convolution module is connected to the input end of the third activation module; the output end of the third activation module is connected to the input end of the twelfth convolution module; the output end of the twelfth convolution module is connected to the input end of the connection module; the output end of the connection module is configured as the model output end of the brain-computer processing model.

[0108] It should be noted that the third activation module is constructed based on the RELU activation function, which is not limited in the embodiment of the present invention.

[0109] It should be noted that the fifth normalization module is constructed based on the batch normalization layer, which is not limited in the embodiment of the present invention.

[0110] It should be noted that the above connection module is constructed based on a one-dimensional fully connected layer, which is not limited in the embodiment of the present invention.

[0111] It should be noted that the convolution kernel size of the ninth convolution module is 60×188, which is not limited in the embodiment of the present invention.

[0112] It should be noted that the convolution kernel sizes of the tenth convolution module, the eleventh convolution module and the twelfth convolution module are 1×120, which is not limited in the embodiment of the present invention.

[0113] It can be seen that implementing the data processing method for drone inspection described in the embodiment of the present invention is conducive to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of a lighter workload, and reducing the manpower cost of power system line inspection.

[0114] Embodiment 2

[0115] See also Figure 3 , Figure 3 : is a schematic diagram of a data processing device for drone inspection disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 3 As shown, the device may include:

[0116] The acquisition module 201 is used to control the drone to acquire inspection image information;

[0117] The processing module 202 is used to perform image segmentation processing on the inspection image information using the first processing terminal to obtain target processed image information;

[0118] The determination module 203 is used to determine the target control signal information based on the second processing terminal and the target processing image information.

[0119] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0120] In another optional embodiment, Figure 3 As shown, the processing module 202 uses the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information, including:

[0121] Receiving inspection image information sent by the drone at the first processing terminal to obtain received image information;

[0122] pre-processing the received image information at the first processing terminal to obtain first processed image information;

[0123] At the first processing terminal, the first processed image information is segmented using a target image processing model to obtain target processed image information.

[0124] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0125] In yet another optional embodiment, Figure 3 As shown, the target image processing model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a thirteenth convolution module, a first normalization module, a second normalization module, a third normalization module, a fourth normalization module, a first upsampling module, a second upsampling module, a first fusion module, a first pooling module, a first multidimensional feature extraction module, a second multidimensional feature extraction module, a third multidimensional feature extraction module, a fourth multidimensional feature extraction module, a first deep feature extraction module, a second deep feature extraction module and a third deep feature extraction module; wherein,

[0126] The input end of the first convolution module is configured as the model input end of the target image processing model, the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the second normalization module; the output end of the second normalization module is connected to the input end of the first multidimensional feature extraction module; the output end of the first multidimensional feature extraction module is respectively connected to the input end of the first deep feature extraction module and the input end of the seventh convolution module; the output end of the first deep feature extraction module is connected to the input end of the second multidimensional feature extraction module; the output end of the second multidimensional feature extraction module is connected to the input end of the second deep feature extraction module; the output end of the second deep feature extraction module is connected to the input end of the third multidimensional feature extraction module; the output end of the third multidimensional feature extraction module is connected to the input end of the third deep feature extraction module; the output end of the third deep feature extraction module is connected to the input end of the fourth multidimensional feature extraction module; the output end of the fourth multidimensional feature extraction module is connected to the thirteenth convolution module The input end of the block; the output end of the thirteenth convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the first pooling module; the output end of the first pooling module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first upsampling module; the output end of the first upsampling module is connected to the input end of the first fusion module; the output end of the seventh convolution module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the fourth normalization module; the output end of the fourth normalization module is connected to the input end of the second activation module; the output end of the second activation module is connected to the input end of the eighth convolution module; the output end of the eighth convolution module is connected to the input end of the second upsampling module; the output end of the second upsampling module is configured as the model output end of the target image processing model.

[0127] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0128] In yet another optional embodiment, Figure 3 As shown, the first multidimensional feature extraction module includes a classification network, a first pooling unit, a first connection unit, a second connection unit, a first activation unit, a first fusion unit, a second fusion unit, a third fusion unit, a fourth fusion unit, a first convolution unit, a second convolution unit, a third convolution unit, a feedforward neural network, a first normalization unit, a second normalization unit, a first dimension conversion unit, a second dimension conversion unit, and a multi-head attention unit; wherein,

[0129] The input end of the classification network is configured as the input end of the first multidimensional feature extraction module, and the output end of the classification network is respectively connected to the input end of the first pooling unit and the input end of the first fusion unit; the output end of the first pooling unit is connected to the input end of the first connection unit; the output end of the first connection unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the second connection unit; the output end of the second connection unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is respectively connected to the input end of the first convolution unit and the input end of the fourth fusion unit; the output end of the first convolution unit is connected to the input end of the first convolution unit; the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the first dimension conversion unit; the first dimension conversion The output end of the conversion unit is respectively connected to the input end of the first normalization unit and the input end of the second fusion unit; the output end of the first normalization unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is respectively connected to the input end of the second normalization unit and the input end of the third fusion unit; the output end of the second normalization unit is connected to the input end of the feedforward neural network; the output end of the feedforward neural network is connected to the input end of the third fusion unit; the output end of the third fusion unit is connected to the input end of the second dimension conversion unit; the output end of the second dimension conversion unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth fusion unit; the output end of the fourth fusion unit is configured as the output end of the first multidimensional feature extraction module.

[0130] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0131] In yet another optional embodiment, Figure 3 As shown, the first deep feature extraction module includes a fourth convolution unit, a fifth convolution unit, a third normalization unit and a second activation unit; wherein,

[0132] The input end of the fourth convolution unit is configured as the input end of the first deep feature extraction module, the output end of the fourth convolution unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second activation unit; the output end of the second activation unit is configured as the output end of the first deep feature extraction module.

[0133] It can be seen that implementation Figure 3The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0134] In yet another optional embodiment, Figure 3 As shown, the determination module 203 determines the target control signal information based on the second processing terminal and the target processing image information, including:

[0135] Displaying target processed image information at a second processing terminal;

[0136] During the signal acquisition period, the second processing terminal is used to acquire brain wave signal information; the brain wave signal information represents a brain wave signal of a continuous time period; the signal acquisition period represents the acquisition time period from when the second processing terminal starts to display the target processing image information to when the second processing terminal completes the acquisition of the brain wave signal;

[0137] The brain wave signal information is processed by using the brain-computer processing model at the second processing terminal to obtain the target control signal information.

[0138] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0139] In yet another optional embodiment, Figure 3 As shown, the brain-computer processing model includes a ninth convolution module, a tenth convolution module, an eleventh convolution module, a twelfth convolution module, a fifth normalization module, a third activation module and a connection module; wherein,

[0140] The ninth convolution module is configured as the model input end of the brain-computer processing model, the output end of the ninth convolution module is connected to the input end of the tenth convolution module; the output end of the tenth convolution module is connected to the input end of the fifth normalization module; the output end of the fifth normalization module is connected to the input end of the eleventh convolution module; the output end of the eleventh convolution module is connected to the input end of the third activation module; the output end of the third activation module is connected to the input end of the twelfth convolution module; the output end of the twelfth convolution module is connected to the input end of the connection module; the output end of the connection module is configured as the model output end of the brain-computer processing model.

[0141] It can be seen that implementation Figure 3 The described data processing device for drone inspection is beneficial to improving the control efficiency of drone inspection and the line inspection control accuracy, realizing the use of drone line inspection under the premise of lighter workload, and reducing the manpower cost of power system line inspection.

[0142] Embodiment 3

[0143] See also Figure 4 , Figure 4 is a structural diagram of another data processing device for drone inspection disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 4 As shown, the device may include:

[0144] A memory 301 storing executable program codes;

[0145] a processor 302 coupled to the memory 301;

[0146] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the data processing method for drone inspection described in the first embodiment.

[0147] Embodiment 4

[0148] An embodiment of the present invention discloses a computer-readable storage medium, which stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the data processing method for drone inspection described in Embodiment 1.

[0149] Embodiment 5

[0150] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the data processing method for drone inspection described in Example 1.

[0151] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0152] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0153] Finally, it should be noted that the data processing method and device for drone inspection disclosed in the embodiment of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data processing method for drone inspection, characterized in that: The method comprises: Control the drone to obtain inspection image information; Using the first processing terminal to perform image segmentation processing on the inspection image information to obtain target processed image information; Based on the second processing terminal and the target processing image information, target control signal information is determined.

2. The data processing method for drone inspection according to claim 1 is characterized in that: The step of performing image segmentation processing on the inspection image information by using the first processing terminal to obtain target processed image information includes: Receiving the inspection image information sent by the drone at the first processing terminal to obtain received image information; pre-processing the received image information at the first processing terminal to obtain first processed image information; At the first processing terminal, the first processed image information is segmented using a target image processing model to obtain target processed image information.

3. The data processing method for drone inspection according to claim 2 is characterized in that: The target image processing model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a thirteenth convolution module, a first normalization module, a second normalization module, a third normalization module, a fourth normalization module, a first upsampling module, a second upsampling module, a first fusion module, a first pooling module, a first multidimensional feature extraction module, a second multidimensional feature extraction module, a third multidimensional feature extraction module, a fourth multidimensional feature extraction module, a first deep feature extraction module, a second deep feature extraction module and a third deep feature extraction module; wherein, The input end of the first convolution module is configured as the model input end of the target image processing model, the output end of the first convolution module is connected to the input end of the first normalization module; the output end of the first normalization module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the second normalization module; the output end of the second normalization module is connected to the input end of the first multidimensional feature extraction module; the output end of the first multidimensional feature extraction module is respectively connected to the input end of the first deep feature extraction module and the input end of the seventh convolution module; the output end of the first deep feature extraction module is connected to the input end of the second multidimensional feature extraction module; the output end of the second multidimensional feature extraction module is connected to the input end of the second deep feature extraction module; the output end of the second deep feature extraction module is connected to the input end of the third multidimensional feature extraction module; the output end of the third multidimensional feature extraction module is connected to the input end of the third deep feature extraction module; the output end of the third deep feature extraction module is connected to the input end of the fourth multidimensional feature extraction module; the output end of the fourth multidimensional feature extraction module is connected to the input end of the thirteenth convolution module. input end; the output end of the thirteenth convolution module is connected to the input end of the third convolution module; the output end of the third convolution module is connected to the input end of the fourth convolution module; the output end of the fourth convolution module is connected to the input end of the fifth convolution module; the output end of the fifth convolution module is connected to the input end of the first pooling module; the output end of the first pooling module is connected to the input end of the sixth convolution module; the output end of the sixth convolution module is connected to the input end of the first up-sampling module; the output end of the first up-sampling module is connected to the input end of the first fusion module; the output end of the seventh convolution module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the first fusion module; the output end of the first fusion module is connected to the input end of the fourth normalization module; the output end of the fourth normalization module is connected to the input end of the second activation module; the output end of the second activation module is connected to the input end of the eighth convolution module; the output end of the eighth convolution module is connected to the input end of the second up-sampling module; the output end of the second up-sampling module is configured as the model output end of the target image processing model.

4. The data processing method for drone inspection according to claim 3 is characterized in that: The first multidimensional feature extraction module includes a classification network, a first pooling unit, a first connection unit, a second connection unit, a first activation unit, a first fusion unit, a second fusion unit, a third fusion unit, a fourth fusion unit, a first convolution unit, a second convolution unit, a third convolution unit, a feedforward neural network, a first normalization unit, a second normalization unit, a first dimension conversion unit, a second dimension conversion unit, and a multi-head attention unit; wherein, The input end of the classification network is configured as the input end of the first multi-dimensional feature extraction module, and the output end of the classification network is respectively connected to the input end of the first pooling unit and the input end of the first fusion unit; the output end of the first pooling unit is connected to the input end of the first connection unit; the output end of the first connection unit is connected to the input end of the first activation unit; the output end of the first activation unit is connected to the input end of the second connection unit; the output end of the second connection unit is connected to the input end of the first fusion unit; the output end of the first fusion unit is respectively connected to the input end of the first convolution unit and the input end of the fourth fusion unit; the output end of the first convolution unit is connected to the input end of the first convolution unit; the output end of the first convolution unit is connected to the input end of the second convolution unit; the output end of the second convolution unit is connected to the input end of the first dimension conversion unit; the first dimension The output end of the degree conversion unit is respectively connected to the input end of the first normalization unit and the input end of the second fusion unit; the output end of the first normalization unit is connected to the input end of the multi-head attention unit; the output end of the multi-head attention unit is connected to the input end of the second fusion unit; the output end of the second fusion unit is respectively connected to the input end of the second normalization unit and the input end of the third fusion unit; the output end of the second normalization unit is connected to the input end of the feedforward neural network; the output end of the feedforward neural network is connected to the input end of the third fusion unit; the output end of the third fusion unit is connected to the input end of the second dimension conversion unit; the output end of the second dimension conversion unit is connected to the input end of the third convolution unit; the output end of the third convolution unit is connected to the input end of the fourth fusion unit; the output end of the fourth fusion unit is configured as the output end of the first multi-dimensional feature extraction module.

5. The data processing method for drone inspection according to claim 3 is characterized in that: The first deep feature extraction module includes a fourth convolution unit, a fifth convolution unit, a third normalization unit and a second activation unit; wherein, The input end of the fourth convolution unit is configured as the input end of the first deep feature extraction module, the output end of the fourth convolution unit is connected to the input end of the third normalization unit; the output end of the third normalization unit is connected to the input end of the fifth convolution unit; the output end of the fifth convolution unit is connected to the input end of the second activation unit; the output end of the second activation unit is configured as the output end of the first deep feature extraction module.

6. The data processing method for drone inspection according to claim 1, characterized in that: The determining of target control signal information based on the second processing terminal and the target processing image information includes: Displaying the target processed image information at the second processing terminal; During a signal acquisition period, the second processing terminal is used to acquire brain wave signal information; the brain wave signal information represents a brain wave signal of a continuous time period; the signal acquisition period represents a collection time period from when the second processing terminal starts to display the target processing image information to when the second processing terminal completes collecting the brain wave signal; The brain wave signal information is processed at the second processing terminal using a brain-computer processing model to obtain target control signal information.

7. The data processing method for drone inspection according to claim 6, characterized in that: The brain-computer processing model includes a ninth convolution module, a tenth convolution module, an eleventh convolution module, a twelfth convolution module, a fifth normalization module, a third activation module and a connection module; wherein, The ninth convolution module is configured as the model input end of the brain-computer processing model, the output end of the ninth convolution module is connected to the input end of the tenth convolution module; the output end of the tenth convolution module is connected to the input end of the fifth normalization module; the output end of the fifth normalization module is connected to the input end of the eleventh convolution module; the output end of the eleventh convolution module is connected to the input end of the third activation module; the output end of the third activation module is connected to the input end of the twelfth convolution module; the output end of the twelfth convolution module is connected to the input end of the connection module; the output end of the connection module is configured as the model output end of the brain-computer processing model.

8. A data processing device for drone inspection, characterized in that: The device comprises: The acquisition module is used to control the UAV to obtain inspection image information; A processing module, used for performing image segmentation processing on the inspection image information by using the first processing terminal to obtain target processed image information; The determination module is used to determine the target control signal information based on the second processing terminal and the target processing image information.

9. A data processing device for drone inspection, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the data processing method for drone inspection as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the data processing method for drone inspection as described in any one of claims 1 to 7.

Images