Multi-light adaptive field-of-view perception fusion method, device and equipment

Abstract

The invention provides a multi-light adaptive view field perception fusion method, device and equipment, and relates to the technical field of image detection, and the method comprises the steps: obtaining a to-be-recognized image; the to-be-recognized image comprises an image of a target object; inputting the to-be-recognized image into a trained neural network model, and determining an anomaly type of the target object according to an output result; and adjusting the working states of the corresponding infrared light image sensor and the ultraviolet light image sensor based on the abnormal type. According to the invention, the abnormal type of the target object is preliminarily judged through the acquired visible light image, and the sensor type required for completing accurate detection is determined according to the abnormal type, so that the infrared light image sensor and / or the ultraviolet light image sensor in the non-working state are / is switched to the working state; the infrared light image sensor and the ultraviolet light image sensor do not need to be kept in a working state in real time, so that the overall power consumption is reduced, and the endurance time is delayed.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a multi-light adaptive field of view perception fusion method, device and equipment. Background Art

[0002] At present, the technologies for partial discharge detection of electrical equipment include observation method, ultra-high frequency method, ultrasonic method, infrared imaging method, optical measurement method, insulating oil chromatography analysis method, ultraviolet imaging method, etc. Among them, infrared thermal imaging technology is increasingly widely used in my country's power system, and it has become an essential means to carry out electrical equipment status inspection. Ultraviolet imaging method is an emerging technology that identifies the insulation status of power equipment by detecting corona and arc discharge. The discharge corona characteristic quantity or temperature infrared thermal radiation of the fault can be effectively detected through the corresponding sensing equipment.

[0003] The drone system has the advantages of strong mobility, agility and safety. It can complete the corona discharge and infrared thermal radiation image shooting tasks by carrying ultraviolet, infrared and visible light optical detection equipment. However, these sensors consume a lot of energy when working simultaneously for a long time, and the energy carried by the drone is limited. How to ensure the smooth progress of the shooting task without adding extra burden to the drone has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the energy consumption of multiple types of sensors on current drones working simultaneously for a long time is extremely high, while the energy carried by the drone is limited. The present invention provides a multi-light adaptive field of view perception fusion method, device and equipment.

[0005] The technical solution of the embodiment of the present application is implemented as follows: A first aspect of an embodiment of the present application provides a multi-light adaptive field of view perception fusion method, including: Acquire an image to be identified; the image to be identified includes an image of a target object; Inputting the image to be identified into a trained neural network model, and determining the abnormal type of the target object according to the output result; The working states of the corresponding infrared light image sensor and ultraviolet light image sensor are adjusted based on the abnormality type.

[0006] Optionally, the target object includes a transformer, an insulator string or a current transformer; and acquiring the image to be identified includes: The image to be identified is acquired using a visible light shooting device.

[0007] Optionally, before inputting the image to be identified into a trained neural network model and determining the abnormal type of the target object according to the output result, the method further includes: Acquire sample images; the sample images include a training set and a test set; The neural network model is trained using the training set, and the neural network model is tested using the test set, until the loss function of the neural network model meets the preset requirements, thereby obtaining a trained neural network model.

[0008] Optionally, the neural network model is a YOLOv8 neural network model.

[0009] Optionally, the abnormality type includes local discharge and local abnormal high temperature; and adjusting the corresponding working states of the infrared image sensor and the ultraviolet image sensor based on the abnormality type includes: When the abnormality type is partial discharge, the ultraviolet image sensor and the visible light image sensor in the visible light shooting device are adjusted to be turned on; When the abnormality type is abnormally high temperature locally, adjusting the infrared light image sensor and the visible light image sensor to an on state; When the abnormality type is local discharge and local abnormal high temperature, the ultraviolet light image sensor, the infrared light image sensor and the visible light image sensor are all adjusted to an on state.

[0010] Optionally, after adjusting the working states of the corresponding infrared image sensor and ultraviolet image sensor based on the abnormality type, the method further includes: Acquire a visible light image detected by the visible light image sensor, an infrared light image detected by the infrared light image sensor, and an ultraviolet light image detected by the ultraviolet light image sensor; The visible light image, the infrared light image and / or the ultraviolet light image are converted into a target output format.

[0011] A second aspect of the embodiment of the present application provides a multi-light adaptive field of view perception fusion device, including: an acquisition module, a determination module and an adjustment module, wherein: The acquisition module is configured to acquire an image to be identified; the image to be identified includes an image of a target object; The determination module is configured to input the image to be identified into a trained neural network model and determine the abnormal type of the target object according to the output result; The adjustment module is configured to adjust the working states of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormality type.

[0012] A third aspect of an embodiment of the present application provides a drone, comprising: a drone body; the drone body is provided with the multi-light adaptive field of view perception fusion device described in the second aspect.

[0013] A fourth aspect of an embodiment of the present application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the multi-light adaptive field of view perception fusion method described in the first aspect.

[0014] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] Compared with the prior art, the technical solution provided by this application has the following beneficial effects: The present invention provides a multi-light adaptive field of view perception fusion method, device and equipment, which obtains the image to be identified, inputs the image to be identified into a trained neural network model, determines the abnormal type of the target object according to the output result; and adjusts the working state of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormal type. The present invention preliminarily determines the abnormal type of the target object through the acquired visible light image, determines the sensor type required to complete accurate detection according to the abnormal type, thereby switching the infrared light image sensor and / or ultraviolet light image sensor in a non-working state to a working state, and does not need to keep the infrared light image sensor and the ultraviolet light image sensor in a working state in real time, thereby reducing the overall power consumption and delaying the battery life. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of a multi-light adaptive field of view perception fusion method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a multi-light adaptive field of view perception and fusion device provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] Below, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application. The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components. All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0018] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0019] In some embodiments, see Figure 1 , Figure 1 A schematic diagram of a flow chart of a multi-light adaptive field of view perception fusion method provided in an embodiment of the present application; the multi-light adaptive field of view perception fusion method provided in an embodiment of the present application includes: S110, obtaining an image to be identified; the image to be identified includes an image of a target object.

[0020] Here, a visible light image sensor is used to obtain multiple frames of images to be identified, wherein the target object may be an abnormally frequent device or an abnormally frequent area on the line.

[0021] S120, inputting the image to be identified into the trained neural network model, and determining the abnormal type of the target object according to the output result.

[0022] In this embodiment, the neural network model can extract features from the image to be identified, and then compare the extracted features with the features in the preset feature database to determine the abnormal type of the target object. It should be noted that the neural network model can output one or more prediction results, which may be the probability distribution of the abnormal type. If only one prediction result is output, the abnormal type of the target object can be directly determined; if multiple prediction results are output, the abnormal type of the target object is determined based on the top three prediction results with the highest probability. It should be noted that this is only an exemplary description. In the case of outputting multiple prediction results, the abnormal type of the target object can also be determined based on 1, 2 or 5 prediction results with the highest probability. The specific determination basis can be determined according to actual needs and is not limited to this.

[0023] S130, adjusting the working states of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormality type.

[0024] There are differences in the application scenarios of infrared image sensors and ultraviolet image sensors. Infrared image sensors are based on the principle of blackbody radiation. They measure the infrared radiation energy emitted by the target object and determine the temperature distribution of the object through photoelectric conversion, signal processing and other means. Ultraviolet image sensors are based on ultraviolet radiation generated by corona discharge. They diagnose the status of electrical equipment by detecting the intensity and distribution of ultraviolet light. Therefore, after determining the type of abnormality, the corresponding sensor can be switched from a non-working state to a working state, or from a low-power working state to a normal-power working state, to achieve further detection of the target object.

[0025] In some embodiments, the target object includes a transformer, an insulator string, or a current transformer; S110, acquiring an image to be identified, including: Use visible light shooting equipment to obtain the image to be identified.

[0026] When a transformer, insulator string or current transformer fails, some will discharge locally and radiate ultraviolet light, while others will cause abnormally high temperatures and radiate infrared light. By using visible light photography equipment to obtain the image to be identified, the approximate type of fault can be determined. On this basis, more accurate fault information can be obtained using the corresponding infrared image sensor or ultraviolet image sensor.

[0027] In some embodiments, at S120, before inputting the image to be identified into the trained neural network model and determining the abnormal type of the target object according to the output result, the multi-light adaptive field of view perception fusion method further includes: Obtain sample images; the sample images include a training set and a test set; The neural network model is trained using the training set, and tested using the test set until the loss function of the neural network model meets the preset requirements, thereby obtaining a trained neural network model.

[0028] In this embodiment, the sample images can be images of various parts of the transmission line under different angles, climates, and lighting conditions, so as to improve the training accuracy of the neural network model. Exemplarily, a large number of labeled images of different parts of the transmission line are prepared to form a data set, which mainly includes partial images of transformers, insulator strings, current transformers, etc., and the images are preprocessed to obtain training sets and test sets for neural network model training and performance testing. Here, preprocessing can include: image transformation and resizing.

[0029] In some embodiments, the neural network model is a YOLOv8 neural network model.

[0030] The YOLOv8 neural network structure can be divided into three main components: Backbone network, Neck network and Head. Backbone network is mainly used to extract multi-level feature images from sample images, and is mainly used in C2f module. Neck network is mainly used for feature enhancement of feature images and feature fusion between multi-level feature images, and is mainly used in SPPF module. Head is mainly used for object detection and object anchor frames in output images. Decoupled Head is used to separate object classification tasks from bounding box regression tasks in two independent branches, which helps to accelerate the convergence process of the model and improve detection accuracy.

[0031] In some embodiments, the abnormality type includes local discharge and local abnormal high temperature; S130, adjusting the working state of the corresponding infrared image sensor and ultraviolet image sensor based on the abnormality type includes: When the abnormality type is partial discharge, the ultraviolet light image sensor and the visible light image sensor in the visible light shooting device are adjusted to be turned on; When the abnormality type is abnormally high temperature locally, the infrared light image sensor and the visible light image sensor are adjusted to be turned on; When the abnormality type is local discharge and local abnormal high temperature, the ultraviolet light image sensor, the infrared light image sensor and the visible light image sensor are all adjusted to the on state.

[0032] In this embodiment, the visible light image sensor can be always in the on state. When the abnormality type is partial discharge, the ultraviolet light image sensor is turned on. When the abnormality type is local abnormal high temperature, the infrared light image sensor is turned on. When the abnormality type includes both partial discharge and local abnormal high temperature, the ultraviolet light image sensor and the infrared light image sensor are turned on at the same time. It should be noted that the abnormality type can also be other situations, which are not enumerated.

[0033] In some embodiments, at S130, after adjusting the working states of the corresponding infrared image sensor and ultraviolet image sensor based on the abnormality type, the multi-light adaptive field of view perception fusion method further includes: Acquire a visible light image detected by a visible light image sensor, an infrared light image detected by an infrared light sensor, and an ultraviolet light image detected by an ultraviolet light image sensor; Convert visible light images, infrared images, and / or ultraviolet images to a target output format.

[0034] In this embodiment, the images obtained by the visible light image sensor, the ultraviolet light image sensor and the infrared light image sensor at the same time are integrated into a unified format and sorted by time, location, etc. to form a data set for subsequent application. Defective images can also be repaired or deleted to ensure the reliability and practicality of the data set.

[0035] In an optional embodiment, the preprocessed sample images are used to train the YOLOv8 neural network model.

[0036] The sample image preprocessing includes: graying the image to eliminate redundant information, converting the color image into a grayscale image, specifically: Gray = 0.299*R+0.578*G+0.114*B; where: R is the red channel component of the image; G is the green channel component of the image; B is the blue channel component of the image; Gray is the grayscale image; using the bilinear interpolation method, adjust the image size to be consistent, specifically: ; in: Represents the value of a two-dimensional function at integer coordinates (i+u, j+v). u and v represent the value of a two-dimensional function at integer coordinates. The offset used to specify the position of the interpolation point. The function values ​​representing four adjacent points are weighted according to certain weights. It means that by interpolating the four points near the integer coordinates (i, j), the function value at the coordinates (i+u,j+v) is obtained.

[0037] After completing the preprocessing of the sample image, the YOLOv8 neural network model is constructed. The neural network model can be divided into three main components: Backbone network, Neck network and Head part. Among them, the Backbone network includes convolution module, C2f module, SPPF module; Neck network includes convolution layer, C2f module and Upsample module; Head part convolution module and loss function calculation. The neural network model is trained and tested using the preprocessed training set and test set until the network converges, and the trained network is further obtained. During the training and testing process, the loss function used by the YOLOv8 neural network model can include classification branch and regression branch.

[0038] The loss function used by the classification branch is BCE loss, and its formula is as follows:

[0039] Where N is the number of samples; yi represents the true label of the i-th sample, which takes a value of 0 or 1; pi represents the probability that the model predicts that the i-th sample is a positive class, which is also a value between 0 and 1; log is the natural logarithm.

[0040] The loss functions used by the regression branch are Distribution Focal Loss and CIoU Loss, and the formulas are as follows:

[0041] in, It is the difficulty sample adjustment term in Focal Loss, γ≥0, and a larger γ will make the model pay more attention to those samples that are difficult to classify (i.e. close to 0 or 1); Represents the predicted probability distribution and the true label distribution The distance metric between them can be KL divergence, JS divergence or other appropriate distribution distance metric methods; α is a balancing factor used to adjust the importance of classification loss and distribution matching loss.

[0042]

[0043] Among them, IoULoss is the basic intersection-over-union loss, which measures the overlap between the predicted box and the real box. CI is the abbreviation of Complete IoU, which is a new part in CIoU Loss, representing the difference between the center point distance and the aspect ratio. The formula is:

[0044] Among them, C is the penalty term for the center point distance, which is calculated as the Euclidean distance of the center point coordinates of the two boxes divided by the diagonal length of the box. α and β are hyperparameters used to adjust the weights of the center point distance and aspect ratio penalty terms. v is the penalty term for the aspect ratio, which is calculated as the absolute value of the difference in aspect ratio between the two boxes. If the aspect ratio difference is large, the penalty term is large.

[0045] After determining the abnormal type of the target object based on the image to be identified, the working status of the infrared image sensor and the ultraviolet image sensor are adjusted dynamically in real time according to the abnormal type. Taking the partial discharge high-risk part set A and the temperature abnormality high-risk part set B as examples, if the transmission line part c in the image only belongs to set A, only the ultraviolet and visible light image sensors are turned on; if the transmission line part c in the image only belongs to set B, only the infrared and visible light image sensors are turned on; if the transmission line part c in the image belongs to both A and B, the ultraviolet, infrared and visible light image sensors are turned on at the same time.

[0046] After that, the images obtained by the visible light image sensor, ultraviolet light image sensor and infrared light image sensor at the same time are processed and integrated into a unified format and sorted by time, location, etc. to form a data set.

[0047] For example, the output format can be The data matrix is ​​, where N is the total number of data groups; 3 is the 3-layer image data, namely visible light, ultraviolet light, and infrared light; W is the uniform width of the image; H is the uniform height of the image; and D is the number of bits of each bit of the image data vector. It should be noted that because the ultraviolet and infrared image sensors are not all working during shooting, if there is no ultraviolet or infrared data, the image data of this layer will be set to 0, indicating that there is no image data. When using this image data later, you can directly use The entire data set can also be used to extract any 2 layers or 1 layer of data required for use, that is, the data set is converted into Format or format, where .

[0048] The embodiment of the present application preliminarily determines the abnormality type of the target object through the acquired visible light image, and determines the sensor type required to complete accurate detection according to the abnormality type, thereby switching the infrared image sensor and / or ultraviolet image sensor in a non-working state to a working state, and there is no need to keep the infrared image sensor and the ultraviolet image sensor in a working state in real time, thereby reducing overall power consumption and extending battery life.

[0049] In some embodiments, see Figure 2 , Figure 2 The schematic diagram of the structure of a multi-light adaptive field of view perception fusion device provided in an embodiment of the present application. The multi-light adaptive field of view perception fusion device 200 provided in an embodiment of the present application includes: an acquisition module 210, a determination module 220 and an adjustment module 230, wherein: The acquisition module 210 is configured to acquire an image to be identified; the image to be identified includes an image of a target object; The determination module 220 is configured to input the image to be identified into the trained neural network model and determine the abnormal type of the target object according to the output result; The adjustment module 230 is configured to adjust the working states of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormality type.

[0050] In some embodiments, the target object includes a transformer, an insulator string, or a current transformer; the acquisition module 210 is specifically configured as follows: Use visible light shooting equipment to obtain the image to be identified.

[0051] In some embodiments, the multi-light adaptive field of view perception fusion device 200 further includes a training module, and the training module is specifically configured as follows: Obtain sample images; the sample images include a training set and a test set; The neural network model is trained using the training set, and tested using the test set until the loss function of the neural network model meets the preset requirements, thereby obtaining a trained neural network model.

[0052] In some embodiments, the neural network model is a YOLOv8 neural network model.

[0053] In some embodiments, the abnormal types include local discharge and local abnormal high temperature; the adjustment module 230 is specifically configured as follows: When the abnormality type is partial discharge, the ultraviolet light image sensor and the visible light image sensor in the visible light shooting device are adjusted to be turned on; When the abnormality type is abnormally high temperature locally, the infrared light image sensor and the visible light image sensor are adjusted to be turned on; When the abnormality type is local discharge and local abnormal high temperature, the ultraviolet light image sensor, the infrared light image sensor and the visible light image sensor are all adjusted to the on state.

[0054] In some embodiments, the multi-light adaptive field of view perception fusion device 200 further includes a format conversion module, and the format conversion module is specifically configured as follows: Acquire a visible light image detected by a visible light image sensor, an infrared light image detected by an infrared light sensor, and an ultraviolet light image detected by an ultraviolet light image sensor; Convert visible light images, infrared images, and / or ultraviolet images to a target output format.

[0055] The multi-light adaptive field of view perception fusion device provided in the embodiment of the present application can implement each process in the embodiment corresponding to the above-mentioned multi-light adaptive field of view perception fusion method. To avoid repetition, it will not be repeated here.

[0056] It should be noted that the multi-light adaptive field of view perception fusion device provided in the embodiment of the present application and the multi-light adaptive field of view perception fusion method provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned multi-light adaptive field of view perception fusion method, and the repeated parts will not be repeated.

[0057] The present application also provides a drone, including: a drone body; the drone body is provided with the above-mentioned multi-light adaptive field of view perception fusion device, and the specific implementation can refer to the implementation of the aforementioned multi-light adaptive field of view perception fusion method, and the repeated parts will not be repeated.

[0058] In some embodiments, see Figure 3 , Figure 3 The electronic device 300 provided in the embodiment of the present application includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program implements the above-mentioned multi-light adaptive field of view perception fusion method when executed by the processor. Specifically, the processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 310 may also include an onboard memory for cache purposes. The processor 310 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiment of the present application. The memory 320 may be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, the memory 320 may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, component or propagation medium. Specific examples of the memory 320 include: a magnetic storage device, such as a magnetic tape or a hard disk (HDD); an optical storage device, such as a compact disk (CD-ROM); a random access memory (RAM) or a flash memory; and / or a wired / wireless communication link. The present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned multi-light adaptive field of view perception fusion method is implemented. The computer-readable medium may be included in the device / apparatus / system described in the above-mentioned embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed, the method according to the embodiment of the present application is implemented. According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above. Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A multi-light adaptive field of view perception fusion method, characterized in that: include: Acquire an image to be identified; the image to be identified includes an image of a target object; Inputting the image to be identified into a trained neural network model, and determining the abnormal type of the target object according to the output result; The working states of the corresponding infrared light image sensor and ultraviolet light image sensor are adjusted based on the abnormality type.

2. The multi-light adaptive field of view perception fusion method according to claim 1, characterized in that: The target object includes a transformer, an insulator string or a current transformer; and obtaining the image to be identified includes: The image to be identified is acquired using a visible light shooting device.

3. The multi-light adaptive field of view perception fusion method according to claim 1, characterized in that: Before inputting the image to be identified into the trained neural network model and determining the abnormal type of the target object according to the output result, the method further includes: Acquire sample images; the sample images include a training set and a test set; The neural network model is trained using the training set, and the neural network model is tested using the test set, until the loss function of the neural network model meets the preset requirements, thereby obtaining a trained neural network model.

4. The multi-light adaptive field of view perception fusion method according to claim 1, characterized in that: The neural network model is a YOLOv8 neural network model.

5. The multi-light adaptive field of view perception fusion method according to claim 1, characterized in that: The abnormality type includes local discharge and local abnormal high temperature; and adjusting the working state of the corresponding infrared image sensor and ultraviolet image sensor based on the abnormality type includes: When the abnormality type is partial discharge, the ultraviolet image sensor and the visible light image sensor in the visible light shooting device are adjusted to be turned on; When the abnormality type is abnormally high temperature locally, adjusting the infrared light image sensor and the visible light image sensor to an on state; When the abnormality type is local discharge and local abnormal high temperature, the ultraviolet light image sensor, the infrared light image sensor and the visible light image sensor are all adjusted to an on state.

6. The multi-light adaptive field of view perception fusion method according to claim 5, characterized in that: After adjusting the working states of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormality type, the method further includes: Acquire a visible light image detected by the visible light image sensor, an infrared light image detected by the infrared light image sensor, and an ultraviolet light image detected by the ultraviolet light image sensor; The visible light image, the infrared light image and / or the ultraviolet light image are converted into a target output format.

7. A multi-light adaptive field of view perception fusion device, characterized in that: include: Acquisition module, determination module and adjustment module, wherein, The acquisition module is configured to acquire an image to be identified; the image to be identified includes an image of a target object; The determination module is configured to input the image to be identified into a trained neural network model and determine the abnormal type of the target object according to the output result; The adjustment module is configured to adjust the working states of the corresponding infrared light image sensor and ultraviolet light image sensor based on the abnormality type.

8. A drone, characterized in that: include: UAV body; the UAV body is provided with the multi-light adaptive field of view perception fusion device described in claim 7.

9. An electronic device comprising a processor and a memory; the memory stores a computer program, wherein: When the computer program is executed by the processor, the computer program implements the multi-light adaptive field of view perception fusion method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images