Target temperature identification method and device based on bimodal data, computer equipment and readable storage medium

Abstract

The invention relates to a target temperature identification method and device based on bimodal data, computer equipment and a readable storage medium. The method comprises the following steps: in response to an inspection instruction for a target area in a power scene, obtaining a bimodal isolation switch image shot by an unmanned aerial vehicle for the target area; inputting the bimodal isolation switch image into a pre-trained target detection model, and outputting a target detection result; the target detection result comprises an isolating switch positioning prediction frame and switch prediction category information corresponding to the isolating switch; inputting a temperature region image containing a temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result; performing optimization processing on the temperature data identification result and the target detection result to obtain interface calling data for the target area; and according to the interface calling data, displaying an isolation switch inspection page of the target area. By adopting the method, the intelligence of showing the inspection result of the disconnecting switch can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for identifying a target temperature based on dual-modal data. Background Art

[0002] With the continuous expansion of the scale of power grids and the development of intelligent trends, the security of power grids is also facing new challenges. Inspection work at all links of the power grid plays a vital role in ensuring the safe and normal operation of the power grid. Substations are key nodes in power transmission. Whether substation equipment, such as disconnectors, can operate normally directly affects the power supply of the power grid.

[0003] As the most widely used and installed equipment in the power grid system, the operating status of the isolating switch is one of the key factors that determine whether the substation can operate safely. Once the isolating switch fails, it will cause serious failures and economic losses. Therefore, during the inspection process, whether the isolating switch is working properly is an important inspection task.

[0004] However, in the related art, when displaying the inspection results of the substation equipment, the displayed content is often relatively simple, which makes it impossible for users to fully understand the situation of the isolating switch.

[0005] Therefore, the inspection result display of the isolating switch in the related art has the problem of low intelligence. Summary of the invention

[0006] Based on this, it is necessary to provide a target temperature identification method, device, computer equipment, computer-readable storage medium and computer program product based on dual-modal data that can improve the intelligence of the inspection result display of the disconnector in order to address the above technical problems.

[0007] In a first aspect, the present application provides a target temperature recognition method based on bimodal data, comprising:

[0008] In response to an inspection instruction for a target area in a power scene, a dual-mode isolating switch image captured by a drone for the target area is acquired; the dual-mode isolating switch image includes a visible light isolating switch image and an infrared isolating switch image;

[0009] Inputting the dual-mode isolating switch image into a pre-trained target detection model, and outputting a target detection result; the target detection result includes an isolating switch location prediction frame and switch prediction category information corresponding to the isolating switch in the isolating switch location prediction frame;

[0010] Inputting a temperature region image containing temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result for the temperature text;

[0011] Optimizing the temperature data recognition result and the target detection result to obtain interface call data for the target area;

[0012] According to the interface call data returned by calling the inspection result query interface, the isolating switch inspection page of the target area is displayed; the isolating switch inspection page includes the infrared isolating switch image containing the target detection result and the equipment temperature identification result, and the inspection result text information; the equipment temperature identification result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0013] In one embodiment, the interface call data includes the switch temperature data, and the temperature text is used to record the temperature data of the infrared isolation switch image; the temperature data recognition result and the target detection result are optimized to obtain the interface call data for the target area, including:

[0014] Determine a mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image;

[0015] According to the regional color data corresponding to the isolating switch area in the infrared isolating switch image and the mapping relationship, the switch temperature data corresponding to the isolating switch is determined; the isolating switch area is the area corresponding to the isolating switch positioning prediction box in the infrared isolating switch image.

[0016] In one embodiment, the temperature data includes a maximum value of the temperature data and a minimum value of the temperature data, and the temperature data recognition result includes a maximum value recognition result of the temperature data and a minimum value recognition result of the temperature data; and determining a mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image includes:

[0017] Converting the infrared isolation switch image into a grayscale image, and determining the image grayscale value of the grayscale image; the image grayscale value is used to characterize the image color data; the image grayscale value includes a maximum grayscale value and a minimum grayscale value;

[0018] Establishing a mapping relationship between the temperature data and the image grayscale value according to the maximum grayscale value and the minimum grayscale value, and the highest value recognition result of the temperature data and the lowest value recognition result of the temperature data;

[0019] The mapping relationship between the temperature data and the image grayscale value is used as the mapping relationship between the temperature data and the image color data.

[0020] In one embodiment, determining the switch temperature data corresponding to the isolating switch according to the regional color data corresponding to the isolating switch region in the infrared isolating switch image and the mapping relationship includes:

[0021] Obtaining a regional image grayscale value corresponding to the isolating switch region; the regional image grayscale value is used to characterize the regional color data; the regional image grayscale value includes at least one of a regional maximum grayscale value, a regional minimum grayscale value, and a regional average grayscale value;

[0022] According to the mapping relationship, the grayscale value of the regional image is converted into corresponding temperature data to obtain the switch temperature data.

[0023] In one embodiment, the displaying of the isolating switch inspection page of the target area according to the interface call data returned by calling the inspection result query interface includes:

[0024] In response to the inspection result query instruction for the target area, the interface call data of the target area is called by calling the inspection result query interface; the interface call data includes the infrared isolation switch image including the target detection result and the device temperature identification result and the inspection result text information;

[0025] According to the interface call data, the isolation switch inspection page is displayed.

[0026] In one embodiment, the temperature text is used to record at least one of the temperature data of the infrared isolation switch image, including the highest temperature data value, the lowest temperature data value and the average temperature data value;

[0027] The step of inputting the temperature region image containing the temperature text in the infrared isolation switch image into a pre-trained text recognition model and outputting a temperature data recognition result for the temperature text comprises:

[0028] The temperature area image is input into the pre-trained text recognition model, and the temperature text in the temperature area image is subjected to text recognition by the pre-trained text recognition model, and at least one temperature data recognition result among the highest temperature data recognition result, the lowest temperature data recognition result and the average temperature data recognition result is output.

[0029] In a second aspect, the present application also provides a target temperature recognition device based on bimodal data, comprising:

[0030] An acquisition module is used to respond to an inspection instruction for a target area in a power scene and acquire a dual-mode disconnector image taken by a drone for the target area; the dual-mode disconnector image includes a visible light disconnector image and an infrared disconnector image;

[0031] A detection module, used to input the dual-mode isolating switch image into a pre-trained target detection model and output a target detection result; the target detection result includes an isolating switch positioning prediction frame and switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction frame;

[0032] A recognition module, used for inputting a temperature region image containing temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result for the temperature text;

[0033] An optimization module, used for optimizing the temperature data recognition result and the target detection result to obtain interface call data for the target area;

[0034] A display module is used to display the isolating switch inspection page of the target area according to the interface call data returned by calling the inspection result query interface; the isolating switch inspection page includes the infrared isolating switch image containing the target detection result and the equipment temperature identification result, and the inspection result text information; the equipment temperature identification result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0035] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0037] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0038] The above-mentioned target temperature recognition method, device, computer equipment, computer-readable storage medium and computer program product based on dual-modal data obtain the dual-modal disconnect switch image taken by the drone for the target area by responding to the inspection instruction for the target area in the power scene; the dual-modal disconnect switch image includes a visible light disconnect switch image and an infrared disconnect switch image; the dual-modal disconnect switch image is input into the pre-trained target detection model, and the target detection result is output; the target detection result includes the disconnect switch positioning prediction box and the switch prediction category information corresponding to the disconnect switch in the disconnect switch positioning prediction box; the temperature area containing temperature text in the infrared disconnect switch image The image is input into a pre-trained text recognition model, and the temperature data recognition result for the temperature text is output; the temperature data recognition result and the target detection result are optimized to obtain the interface call data for the target area; according to the interface call data returned by the inspection result query interface, the isolating switch inspection page of the target area is displayed; the isolating switch inspection page includes an infrared isolating switch image containing the target detection result and the equipment temperature recognition result, as well as the inspection result text information; the equipment temperature recognition result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0039] In this way, the dual-mode isolating switch image is input into the pre-trained target detection model, and the pre-trained target detection model can fuse the features corresponding to the visible light isolating switch image and the features corresponding to the infrared isolating switch image, and accurately output the target detection result including the isolating switch positioning prediction box and the switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction box. The temperature area image including the temperature text in the infrared isolating switch image is input into the pre-trained text recognition model, and the temperature text can be more accurately identified to obtain the temperature data recognition result for the temperature text. By optimizing the temperature data recognition result and the target detection result, the interface call data of the target area is obtained, so that when the isolating switch inspection page of the target area is displayed according to the interface call data returned by the query interface for calling the inspection result, the infrared isolating switch image including the target detection result and the equipment temperature recognition result, as well as the inspection result text information including the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data can be displayed, so as to realize the display of the inspection result of the isolating switch in the target area from multiple aspects of image and text, so that the user can fully understand the situation of the isolating switch, and the intelligence of the display of the inspection result of the isolating switch is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a flow chart of a method for identifying target temperature based on dual-modal data in one embodiment;

[0042] Figure 2 is a flow chart of yet another method for identifying target temperature based on dual-modal data in an embodiment;

[0043] Figure 3 is a flow chart of a method for identifying target temperature based on dual-modal data in another embodiment;

[0044] Figure 4 is a structural block diagram of a target temperature identification device based on dual-modal data in one embodiment;

[0045] FIG. 5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, Figure 1 As shown, a method for identifying target temperature based on bimodal data is provided. This embodiment uses the method applied to a computer device as an example. It can be understood that the computer device can be a terminal or a system including a terminal and a server. In this embodiment, the method includes the following steps:

[0048] Step S110, in response to an inspection instruction for a target area in a power scene, an image of a dual-mode disconnector captured by a drone for the target area is acquired.

[0049] The dual-mode isolation switch image includes a visible light isolation switch image and an infrared isolation switch image.

[0050] The target area includes an isolating switch to be tested.

[0051] The visible light isolating switch image may be a visible light image including the isolating switch.

[0052] The infrared isolating switch image may be an infrared image including the isolating switch.

[0053] Among them, the inspection command is used to instruct the drone to take pictures of the target area.

[0054] In a specific implementation, a computer device can provide an inspection interface that matches the power scene. A user account can select an area to be inspected in the inspection interface to input an inspection instruction for the area to the computer device. The area selected by the user account is used as a target area. The computer device responds to the inspection instruction for the target area in the power scene and issues an inspection task for the target area to the drone. The drone can then photograph the target area using the visible light camera and infrared camera it carries. The target area contains an isolating switch to be tested, so that the computer device can obtain a dual-mode isolating switch image; the dual-mode isolating switch image includes a visible light isolating switch image and an infrared isolating switch image.

[0055] Step S120, inputting the dual-mode isolation switch image into a pre-trained target detection model, and outputting the target detection result.

[0056] Among them, the target detection result includes the isolating switch positioning prediction box and the switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction box.

[0057] The switch prediction category information includes the category predicted by the pre-trained target detection model for the detected isolation switch.

[0058] Among them, the isolating switch positioning prediction box is used to locate the isolating switch detected in the dual-modal isolating switch image.

[0059] In a specific implementation, a computer device can input a dual-modal isolation switch image into a pre-trained target detection model, perform target detection on the dual-modal isolation switch image through the pre-trained target detection model, and output a target detection result. The target detection result includes an isolation switch positioning prediction box and switch prediction category information corresponding to the isolation switch in the isolation switch positioning prediction box.

[0060] Among them, before the visible light isolation switch image and the infrared isolation switch image are input into the pre-trained target detection model, the corresponding visible light isolation switch image and the infrared isolation switch image can be aligned, and the aligned visible light isolation switch image and the infrared isolation switch image can be input into the pre-trained target detection model for target detection.

[0061] Among them, the pre-trained target detection model includes a backbone network layer (Backbone), a neck network layer (Neck) and a head network layer (Head). By inputting the dual-modal isolation switch image into the backbone network layer, the visible light isolation switch image and the infrared isolation switch image are respectively extracted, and the visible light modal feature map corresponding to the visible light isolation switch image and the infrared modal feature map corresponding to the infrared isolation switch image can be output; by inputting the visible light modal feature map and the infrared modal feature map into the neck network layer, the visible light modal feature map and the infrared modal feature map are feature fused, and the dual-modal fusion feature map is output. The dual-modal fusion feature map is input into the head network layer, and the dual-modal fusion feature map is subjected to target detection, and the target detection result is output.

[0062] In this way, the infrared image and the visible light image are effectively fused, the complementarity of the image features is enhanced, the image quality is improved, and a clearer data basis is provided for the subsequent information identification of the isolating switch, so as to improve the detection accuracy and reduce the false detection rate.

[0063] Step S130, inputting the temperature area image containing the temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result for the temperature text.

[0064] The temperature text is used to record the temperature data of the infrared isolation switch image, for example, to record at least one of the highest temperature data value, the lowest temperature data value and the average temperature data value of the infrared isolation switch image.

[0065] In a specific implementation, a computer device can extract the area containing temperature text from the infrared isolation switch image as a temperature area image, input it into a pre-trained text recognition model to perform text recognition on the temperature text, and output a temperature data recognition result for the temperature text.

[0066] Step S140, optimizing the temperature data recognition result and the target detection result to obtain interface call data for the target area.

[0067] In a specific implementation, the computer device can optimize the temperature data recognition results and the target detection results to obtain the interface call data for the target area.

[0068] For example, since different colors in the infrared isolating switch image represent different temperatures, the computer device can determine the switch temperature data corresponding to the isolating switch in the isolating switch positioning prediction box based on the temperature data recognition result and the color data of the area corresponding to the isolating switch positioning prediction box in the infrared isolating switch image. The interface call data for the target area can include the switch temperature data corresponding to the isolating switch.

[0069] Step S150, displaying the isolating switch inspection page of the target area according to the interface call data returned by the inspection result query interface.

[0070] Among them, the isolating switch inspection page includes an infrared isolating switch image containing target detection results and equipment temperature recognition results, as well as inspection result text information.

[0071] The equipment temperature identification result includes switch temperature data corresponding to the isolating switch.

[0072] The inspection result text information includes at least identification information corresponding to the disconnector, switch prediction category information, and switch temperature data.

[0073] In a specific implementation, the computer device can display the isolating switch inspection page of the target area according to the interface call data returned by calling the inspection result query interface. The isolating switch inspection page includes an infrared isolating switch image including the target detection result and the device temperature identification result, that is, the target detection result and the device temperature identification result are displayed on the infrared isolating switch image. At the same time, the isolating switch inspection page also includes the inspection result text information; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0074] In the above-mentioned target temperature recognition method based on dual-modal data, by responding to the inspection instruction for the target area in the power scene, a dual-modal isolating switch image taken by the drone for the target area is obtained; the dual-modal isolating switch image includes a visible light isolating switch image and an infrared isolating switch image; the dual-modal isolating switch image is input into a pre-trained target detection model, and the target detection result is output; the target detection result includes an isolating switch positioning prediction box and the switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction box; the temperature area image containing temperature text in the infrared isolating switch image is input into a pre-trained text recognition model, and the temperature data recognition result for the temperature text is output; the temperature data recognition result and the target detection result are optimized to obtain the interface call data for the target area; according to the interface call data returned by the call inspection result query interface, the isolating switch inspection page of the target area is displayed; the isolating switch inspection page includes the infrared isolating switch image including the target detection result and the equipment temperature recognition result, as well as the inspection result text information; the equipment temperature recognition result includes the switch temperature data corresponding to the isolating switch; the inspection result text information at least includes the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0075] In this way, the dual-mode isolating switch image is input into the pre-trained target detection model, and the pre-trained target detection model can fuse the features corresponding to the visible light isolating switch image and the features corresponding to the infrared isolating switch image, and accurately output the target detection result including the isolating switch positioning prediction box and the switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction box. The temperature area image including the temperature text in the infrared isolating switch image is input into the pre-trained text recognition model, and the temperature text can be more accurately identified to obtain the temperature data recognition result for the temperature text. By optimizing the temperature data recognition result and the target detection result, the interface call data of the target area is obtained, so that when the isolating switch inspection page of the target area is displayed according to the interface call data returned by the query interface for calling the inspection result, the infrared isolating switch image including the target detection result and the equipment temperature recognition result, as well as the inspection result text information including the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data can be displayed, so as to realize the display of the inspection result of the isolating switch in the target area from multiple aspects of image and text, so that the user can fully understand the situation of the isolating switch, and the intelligence of the display of the inspection result of the isolating switch is improved.

[0076] In some embodiments, the interface call data includes switch temperature data, and the temperature text is used to record the temperature data of the infrared isolation switch image; the temperature data recognition results and the target detection results are optimized to obtain the interface call data for the target area, including: determining the mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image; determining the switch temperature data corresponding to the isolation switch according to the regional color data corresponding to the isolation switch area in the infrared isolation switch image, and the mapping relationship; the isolation switch area is the area corresponding to the isolation switch positioning prediction box in the infrared isolation switch image.

[0077] In a specific implementation, the interface call data includes switch temperature data, and the temperature text is used to record the temperature data of the infrared isolation switch image. When the computer device optimizes the temperature data recognition results and the target detection results to obtain the interface call data for the target area, different colors in the infrared isolation switch image represent different temperatures. In this way, the computer device can establish a mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image based on the color data corresponding to the temperature data recognition results in the infrared isolation switch image. Among them, the image color data includes all color data contained in the infrared isolation switch image.

[0078] The computer device can determine the switch temperature data corresponding to the isolating switch according to the regional color data corresponding to the isolating switch area in the infrared isolating switch image and the mapping relationship. The isolating switch area is the area corresponding to the isolating switch positioning prediction box in the infrared isolating switch image. The regional color data is the color data corresponding to the isolating switch area in the infrared isolating switch image.

[0079] In some embodiments, the temperature data includes a maximum temperature data value and a minimum temperature data value, and the temperature data recognition result includes a maximum temperature data value recognition result and a minimum temperature data value recognition result. In the process of determining the mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image, the computer device may convert the infrared isolation switch image into a grayscale image and determine the image grayscale value of the grayscale image; the image grayscale value is used to characterize the image color data; the image grayscale value includes a maximum grayscale value and a minimum grayscale value, and the computer device may establish a mapping relationship between the temperature data and the image grayscale value according to the maximum grayscale value and the minimum grayscale value, as well as the maximum temperature data value recognition result and the minimum temperature data value recognition result; wherein the mapping relationship may be a linear relationship, and the computer device may use the mapping relationship between the temperature data and the image grayscale value as the mapping relationship between the temperature data and the image color data.

[0080] In some embodiments, when the computer device determines the switch temperature data corresponding to the isolating switch according to the regional color data corresponding to the isolating switch region in the infrared isolating switch image and the mapping relationship, the computer device can obtain the regional image grayscale value corresponding to the isolating switch region; wherein the regional image grayscale value is the grayscale value of the isolating switch region in the grayscale image; the regional image grayscale value is used to characterize the regional color data; the regional image grayscale value includes at least one of the regional maximum grayscale value, the regional minimum grayscale value and the regional average grayscale value; thus, the computer device can convert the regional image grayscale value into the corresponding temperature data according to the mapping relationship to obtain the switch temperature data. Thus, the switch temperature data can include at least one of the switch maximum temperature data, the switch minimum temperature data and the switch average temperature data.

[0081] The technical solution of this embodiment is to obtain the regional image grayscale value corresponding to the isolating switch area; the regional image grayscale value is used to characterize the regional color data; the regional image grayscale value includes at least one of the regional maximum grayscale value, the regional minimum grayscale value and the regional average grayscale value; according to the mapping relationship, the regional image grayscale value is converted into the corresponding temperature data, so as to accurately obtain the switch temperature data of the isolating switch.

[0082] In one embodiment, the isolating switch inspection page of the target area is displayed based on the interface calling data returned by calling the inspection result query interface, including: in response to the inspection result query instruction for the target area, calling the interface calling data of the target area by calling the inspection result query interface; the interface calling data includes an infrared isolating switch image containing target detection results and equipment temperature identification results, and inspection result text information; based on the interface calling data, the isolating switch inspection page is displayed.

[0083] In a specific implementation, when the computer device displays the inspection page of the isolating switch of the target area according to the interface call data returned by calling the inspection result query interface, the inspection interface includes an inspection result query entry for the target area, and the user account can input the inspection result query instruction for the target area into the inspection interface by clicking the inspection result query entry corresponding to the target area. In this way, the computer device can call the interface call data of the target area by calling the inspection result query interface, and the interface call data includes the infrared isolating switch image including the target detection result and the device temperature identification result and the inspection result text information. In this way, the computer device can display the inspection page of the isolating switch according to the interface call data.

[0084] The technical solution of this embodiment is to call the interface call data of the target area by calling the inspection result query interface in response to the inspection result query instruction for the target area; the interface call data includes the infrared isolation switch image containing the target detection result and the device temperature identification result and the inspection result text information; according to the interface call data, the isolation switch inspection page is displayed. In this way, through the interface call data, the infrared isolation switch image containing the target detection result and the device temperature identification result and the inspection result text information related to the isolation switch in the target area can be integrated, so that the isolation switch inspection page can be efficiently and accurately displayed according to the interface call data.

[0085] In one embodiment, a temperature area image containing temperature text in an infrared isolation switch image is input into a pre-trained text recognition model, and a temperature data recognition result for the temperature text is output, including: inputting the temperature area image into a pre-trained text recognition model, performing text recognition on the temperature text in the temperature area image through the pre-trained text recognition model, and outputting at least one temperature data recognition result among a maximum temperature data recognition result, a minimum temperature data recognition result, and an average temperature data recognition result.

[0086] The temperature text is used to record at least one of the temperature data of the infrared isolation switch image, including the highest temperature data value, the lowest temperature data value and the average temperature data value.

[0087] In a specific implementation, when a computer device inputs a temperature area image containing temperature text in an infrared isolation switch image into a pre-trained text recognition model and outputs a temperature data recognition result for the temperature text, the computer device may input the temperature area image into a pre-trained text recognition model, perform text recognition on the temperature text in the temperature area image through the pre-trained text recognition model, and output at least one temperature data recognition result among a maximum temperature data recognition result, a minimum temperature data recognition result, and an average temperature data recognition result.

[0088] In order to facilitate the understanding of those skilled in the art, Figure 2 A flow chart of another target temperature recognition method based on dual-modal data is provided. Figure 2 As shown, for the disconnector components of the power scene (for example, 110kv and 500kv disconnectors), a rectangular frame is used to mark the outer contour of the bushing component, and a YOLOv8-target detection model for high-resolution data is trained to obtain a pre-trained target detection model. The detailed implementation method of this step is as follows:

[0089] 1. Select the substation images inspected by drones, filter out infrared images and visible light images with target disconnectors as sample images of visible light disconnectors and infrared disconnectors, and use image annotation tool software to annotate the casing with a rectangular frame;

[0090] 2. Use data augmentation schemes such as Mosaic (data augmentation algorithm), flipping, random image cropping, and random contrast to augment the data set, and divide the training set and test set into a preset ratio (e.g. 8:2). Mosaic randomly uses 9 pictures, randomly scales them, and then splices them with random distribution; in this way, the image enhancement technology is introduced to effectively reduce the impact of environmental noise on image recognition and improve the anti-interference ability;

[0091] 3. The Yolov8 model mainly includes the following modules:

[0092] (1) Provides a brand new SOTA model (including P5 640 and P6 1280 resolution object detection networks and instance segmentation models based on YOLACT (You Only Look At CoefficienTs, a real-time instance segmentation fully convolutional network)). In addition, models of different scales N / S / M / L / X are provided based on the scaling factor to meet the needs of different deployment platforms and application scenarios.

[0093] (2) Backbone: The C2f module is used to achieve further lightweighting, the SPPF (Spatial Pyramid Pooling – Fast) module is used, and models of different scales are carefully fine-tuned, which greatly improves model performance.

[0094] (3) Neck: The 1*1 downsampling layer is removed.

[0095] (4) Head: A decoupled-head structure is used to separate the classification and detection heads, while using Anchor-Free (without anchor frame).

[0096] (5) Loss calculation: VFL Loss (Varifocal Loss) is used as the classification loss (BCE Loss (Binary Cross-Entropy Loss) is used in actual training); DFL Loss (Distribution Focal Loss) and CIOU Loss (Complete IoU Loss) are used as regression losses.

[0097] (6) Label assignment: Use Task-Aligned Assigner to assign positive and negative samples;

[0098] In this way, through machine learning and deep learning techniques, the algorithm is trained to adapt to different environmental conditions and equipment types, which enhances the algorithm's versatility and robustness;

[0099] 4. The pre-trained target detection model trained by the yolov8 target detection method can output the target detection result. Specifically, the pre-trained target detection model can detect the outer frame of each category of isolating switches in the image, that is, the isolating switch positioning prediction frame, and on this basis, merge multiple isolating switch positioning prediction frames in the same image. The isolating switch positioning prediction frame contains information such as [switch prediction category information, prediction box upper left corner x, prediction box upper left corner y, prediction box lower right corner x, prediction box lower right corner y, prediction probability value], and limit an image to have at most 3 isolating switch positioning prediction frames; in this way, the target detection result can include the isolating switch positioning prediction frame, the switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction frame, the isolating switch positioning prediction frame coordinate information and the prediction probability information;

[0100] 5. After obtaining the target detection result, extract the temperature area image containing the temperature text in the infrared isolation switch image, and pass it into the pre-trained text recognition model to output the temperature data recognition result; the temperature text can record the maximum value of the temperature data (max temperature), the minimum value of the temperature data (min temperature) and the average value of the temperature data (avg temperature); the pre-trained text recognition model can be the paddle_ocr (Optical Character Recognition) model; in this way, in view of the particularity of temperature information in infrared images, the temperature data in small fonts and complex backgrounds in the image can be accurately recognized, which significantly improves the recognition accuracy;

[0101] 6. Optimize the temperature data recognition results and target detection results to obtain interface call data. The inspection result text information in the interface call data can be in the following form: such as [switch prediction category information, prediction box upper left corner x, prediction box upper left corner y, prediction box lower right corner x, prediction box lower right corner y, prediction probability value, switch max temperature (switch maximum temperature data), switch min temperature (switch minimum temperature data), switch avg temperature (switch average temperature data)].

[0102] In some embodiments, the identified switch temperature data can be compared with the switch temperature data corresponding to each identified isolating switch by using a preset temperature data threshold, thereby identifying the isolating switch with abnormal temperature data and providing a prompt.

[0103] In this way, this application uses the method of fusing infrared data with visible light data to improve the recognition effect of disconnectors. It achieves enhanced small sample recognition effect: When faced with multi-category disconnector data and uneven sample sizes in each category, a unified labeling standard helps to enhance the recognition effect of small sample category prediction boxes. This method enables the algorithm to learn and generalize better, and maintain a high recognition accuracy even when the sample size of individual categories is small. It achieves the effect of having good scalability:

[0104] This technical solution using rectangular frame annotation is not limited to the current 2 types of disconnectors, but also has good scalability. When a new disconnector needs to be identified, only an appropriate amount of sample data needs to be added to generate a certain recognition capability. This scalability enables the algorithm to flexibly respond to more meter types and recognition requirements that may appear in the future, improving the applicability and advancement of the technology in the long term. In addition, the combination of target detection and optical character recognition can simultaneously realize the functions of target detection and temperature information extraction.

[0105] In another embodiment, Figure 3As shown, a target temperature recognition method based on bimodal data is provided, and the method is applied to a computer device as an example for explanation, and includes the following steps:

[0106] Step S302, in response to an inspection instruction for a target area in a power scene, an image of a dual-mode disconnector captured by a drone for the target area is acquired.

[0107] Step S304: input the dual-mode isolation switch image into a pre-trained target detection model, and output the target detection result.

[0108] Step S306: input the temperature area image containing the temperature text in the infrared isolation switch image into a pre-trained text recognition model, and output a temperature data recognition result for the temperature text.

[0109] Step S308: convert the infrared isolation switch image into a grayscale image, and determine the image grayscale value of the grayscale image.

[0110] Step S310, establishing a mapping relationship between temperature data and image grayscale values ​​according to the maximum grayscale value and the minimum grayscale value, as well as the highest value recognition result of the temperature data and the lowest value recognition result of the temperature data.

[0111] Step S312: The mapping relationship between the temperature data and the image grayscale value is used as the mapping relationship between the temperature data and the image color data.

[0112] Step S314, obtaining the grayscale value of the regional image corresponding to the isolating switch area, and converting the grayscale value of the regional image into corresponding temperature data according to the mapping relationship to obtain the switch temperature data.

[0113] Step S316, in response to the inspection result query instruction for the target area, the interface call data of the target area is called by calling the inspection result query interface.

[0114] Step S318, displaying the isolating switch inspection page according to the interface call data.

[0115] It should be noted that the specific definition of the above steps can refer to the specific definition of a target temperature identification method based on dual-modal data above.

[0116] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0117] Based on the same inventive concept, the embodiment of the present application also provides a target temperature recognition device based on dual-modal data for implementing the target temperature recognition method based on dual-modal data involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the target temperature recognition device based on dual-modal data provided below can refer to the limitations of the target temperature recognition method based on dual-modal data above, and will not be repeated here.

[0118] In an exemplary embodiment, Figure 4 As shown, a target temperature recognition device based on bimodal data is provided, comprising: an acquisition module 410, a detection module 420, an identification module 430, an optimization module 440 and a display module 450, wherein:

[0119] The acquisition module 410 is used to respond to the inspection instruction for the target area in the power scene and obtain the dual-mode isolation switch image taken by the drone for the target area; the dual-mode isolation switch image includes a visible light isolation switch image and an infrared isolation switch image.

[0120] The detection module 420 is used to input the dual-modal isolating switch image into a pre-trained target detection model and output a target detection result; the target detection result includes an isolating switch positioning prediction box and switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction box.

[0121] The recognition module 430 is used to input the temperature area image containing the temperature text in the infrared isolation switch image into a pre-trained text recognition model, and output a temperature data recognition result for the temperature text.

[0122] The optimization module 440 is used to optimize the temperature data recognition result and the target detection result to obtain the interface calling data for the target area.

[0123] The display module 450 is used to display the isolating switch inspection page of the target area according to the interface call data returned by calling the inspection result query interface; the isolating switch inspection page includes the infrared isolating switch image containing the target detection result and the equipment temperature identification result, and the inspection result text information; the equipment temperature identification result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

[0124] In one embodiment, the interface call data includes the switch temperature data, and the temperature text is used to record the temperature data of the infrared isolation switch image; the optimization module 440 is specifically used to determine the mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image; determine the switch temperature data corresponding to the isolation switch according to the regional color data corresponding to the isolation switch area in the infrared isolation switch image and the mapping relationship; the isolation switch area is the area corresponding to the isolation switch positioning prediction box in the infrared isolation switch image.

[0125] In one embodiment, the temperature data includes a maximum temperature data value and a minimum temperature data value, and the temperature data recognition result includes a maximum temperature data value recognition result and a minimum temperature data value recognition result; the optimization module 440 is specifically used to convert the infrared isolation switch image into a grayscale image and determine the image grayscale value of the grayscale image; the image grayscale value is used to characterize the image color data; the image grayscale value includes a maximum grayscale value and a minimum grayscale value; based on the maximum grayscale value and the minimum grayscale value, as well as the maximum temperature data value recognition result and the minimum temperature data value recognition result, a mapping relationship between the temperature data and the image grayscale value is established; the mapping relationship between the temperature data and the image grayscale value is used as the mapping relationship between the temperature data and the image color data.

[0126] In one embodiment, the optimization module 440 is specifically used to obtain the regional image grayscale value corresponding to the isolation switch area; the regional image grayscale value is used to characterize the regional color data; the regional image grayscale value includes at least one of the regional maximum grayscale value, the regional minimum grayscale value and the regional average grayscale value; according to the mapping relationship, the regional image grayscale value is converted into corresponding temperature data to obtain the switch temperature data.

[0127] In one embodiment, the display module 450 is specifically used to respond to the inspection result query instruction for the target area, and call the interface call data of the target area by calling the inspection result query interface; the interface call data includes the infrared isolation switch image containing the target detection result and the equipment temperature identification result and the inspection result text information; according to the interface call data, the isolation switch inspection page is displayed.

[0128] In one embodiment, the temperature text is used to record at least one of the temperature data of the highest value, the temperature data of the lowest value, and the temperature data average value of the infrared isolation switch image; the recognition module 430 is specifically used to input the temperature area image into the pre-trained text recognition model, perform text recognition on the temperature text in the temperature area image through the pre-trained text recognition model, and output at least one temperature data recognition result of the highest temperature data recognition result, the lowest temperature data recognition result, and the temperature data average value recognition result.

[0129] Each module in the above-mentioned target temperature recognition device based on dual-modal data can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0130] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG5. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a target temperature recognition method based on dual-modal data is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0131] Those skilled in the art will understand that the structure shown in FIG. 5 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0132] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0133] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0134] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0136] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0137] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0138] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target temperature recognition method based on bimodal data, It is characterized in that The method comprises: In response to an inspection instruction for a target area in a power scene, a dual-mode isolating switch image captured by a drone for the target area is acquired; the dual-mode isolating switch image includes a visible light isolating switch image and an infrared isolating switch image; Inputting the dual-mode isolating switch image into a pre-trained target detection model, and outputting a target detection result; the target detection result includes an isolating switch location prediction frame and switch prediction category information corresponding to the isolating switch in the isolating switch location prediction frame; Inputting a temperature region image containing temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result for the temperature text; Optimizing the temperature data recognition result and the target detection result to obtain interface call data for the target area; According to the interface call data returned by calling the inspection result query interface, the isolating switch inspection page of the target area is displayed; the isolating switch inspection page includes the infrared isolating switch image containing the target detection result and the equipment temperature identification result, and the inspection result text information; the equipment temperature identification result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

2. The method according to claim 1, It is characterized in that The interface call data includes the switch temperature data, and the temperature text is used to record the temperature data of the infrared isolation switch image; the temperature data recognition result and the target detection result are optimized to obtain the interface call data for the target area, including: Determine a mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image; According to the regional color data corresponding to the isolating switch area in the infrared isolating switch image and the mapping relationship, the switch temperature data corresponding to the isolating switch is determined; the isolating switch area is the area corresponding to the isolating switch positioning prediction box in the infrared isolating switch image.

3. The method according to claim 2, It is characterized in that The temperature data includes a maximum value of the temperature data and a minimum value of the temperature data, and the temperature data recognition result includes a maximum value recognition result of the temperature data and a minimum value recognition result of the temperature data; the determining of a mapping relationship between the temperature data and the image color data corresponding to the infrared isolation switch image according to the color data corresponding to the temperature data recognition result in the infrared isolation switch image includes: Converting the infrared isolation switch image into a grayscale image, and determining the image grayscale value of the grayscale image; the image grayscale value is used to characterize the image color data; the image grayscale value includes a maximum grayscale value and a minimum grayscale value; Establishing a mapping relationship between the temperature data and the image grayscale value according to the maximum grayscale value and the minimum grayscale value, and the highest value recognition result of the temperature data and the lowest value recognition result of the temperature data; The mapping relationship between the temperature data and the image grayscale value is used as the mapping relationship between the temperature data and the image color data.

4. The method according to claim 3, It is characterized in that The step of determining the switch temperature data corresponding to the isolating switch according to the regional color data corresponding to the isolating switch region in the infrared isolating switch image and the mapping relationship comprises: Obtaining a regional image grayscale value corresponding to the isolating switch region; the regional image grayscale value is used to characterize the regional color data; the regional image grayscale value includes at least one of a regional maximum grayscale value, a regional minimum grayscale value, and a regional average grayscale value; According to the mapping relationship, the grayscale value of the regional image is converted into corresponding temperature data to obtain the switch temperature data.

5. The method according to claim 1, It is characterized in that The interface call data returned by calling the inspection result query interface is used to display the inspection page of the isolating switch in the target area, including: In response to the inspection result query instruction for the target area, the interface call data of the target area is called by calling the inspection result query interface; the interface call data includes the infrared isolation switch image including the target detection result and the device temperature identification result and the inspection result text information; According to the interface call data, the isolation switch inspection page is displayed.

6. The method according to claim 1, It is characterized in that The temperature text is used to record at least one of the temperature data of the infrared isolation switch image, the temperature data of the highest value, the temperature data of the lowest value and the temperature data of the average value; The step of inputting the temperature region image containing the temperature text in the infrared isolation switch image into a pre-trained text recognition model and outputting a temperature data recognition result for the temperature text comprises: The temperature area image is input into the pre-trained text recognition model, and the temperature text in the temperature area image is subjected to text recognition by the pre-trained text recognition model, and at least one temperature data recognition result among the highest temperature data recognition result, the lowest temperature data recognition result and the average temperature data recognition result is output.

7. A target temperature recognition device based on bimodal data, It is characterized in that The device comprises: An acquisition module is used to respond to an inspection instruction for a target area in a power scene and acquire a dual-mode disconnector image taken by a drone for the target area; the dual-mode disconnector image includes a visible light disconnector image and an infrared disconnector image; A detection module, used to input the dual-mode isolating switch image into a pre-trained target detection model and output a target detection result; the target detection result includes an isolating switch positioning prediction frame and switch prediction category information corresponding to the isolating switch in the isolating switch positioning prediction frame; A recognition module, used for inputting a temperature region image containing temperature text in the infrared isolation switch image into a pre-trained text recognition model, and outputting a temperature data recognition result for the temperature text; An optimization module, used for optimizing the temperature data recognition result and the target detection result to obtain interface call data for the target area; A display module is used to display the isolating switch inspection page of the target area according to the interface call data returned by calling the inspection result query interface; the isolating switch inspection page includes the infrared isolating switch image containing the target detection result and the equipment temperature identification result, and the inspection result text information; the equipment temperature identification result includes the switch temperature data corresponding to the isolating switch; the inspection result text information includes at least the identification information corresponding to the isolating switch, the switch prediction category information and the switch temperature data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images