Infrared intelligent diagnosis method, device and system based on improved YOLACTT + +

Abstract

The invention discloses an infrared intelligent diagnosis method, device and system based on improved YOLACTT + +, and relates to the technical field of map recognition. Comprising the steps of obtaining a grayscale image of target equipment; a target component in the target equipment is extracted according to the grayscale image of the target equipment, a Mask mask of the target component is extracted through an improved YOLCT + + neural network, a Mask mask area of the target component is obtained, and the improved YOLCT + + neural network is obtained by fusing a YOLOX neural network and a YOLCT + + neural network; extracting effective temperature data of the target component in the Mask mask area; and performing fault diagnosis on the effective temperature data according to the temperature characteristics. According to the method, the advantages of YOLOX and YOLACTT + + are combined, the precision and speed of detection are improved, noise interference is reduced, the efficiency and accuracy of Mask mask extraction are improved, and the robustness and accuracy of detection are further improved.

Description

Technical Field

[0001] The present application belongs to the field of pattern recognition technology, and in particular, relates to an infrared intelligent diagnosis method, device and system based on improved YOLACT++. Background Art

[0002] Image recognition is generally divided into traditional segmentation algorithms and deep learning methods. Commonly used traditional segmentation algorithms include threshold segmentation algorithms, region growing segmentation algorithms, and edge detection segmentation algorithms. Deep learning segmentation algorithms include Mask-RCNN, DeepLab, PSPNet, U-net, etc. The use of Mask-RCNN, Fast-RCNN and other algorithms has achieved certain application results in infrared image recognition, but the above algorithms have an image segmentation speed of hundreds of milliseconds.

[0003] In the inspection process of live infrared detection of power equipment, a large number of infrared images need to be processed in real time. These images have the characteristics of diverse colors, complex backgrounds, changing angles, and image and text interference, which poses higher challenges and brings greater difficulties to the intelligent detection and identification of abnormal heating fault points in infrared images. The methods for infrared image segmentation of power equipment can be roughly divided into traditional segmentation algorithms and deep learning methods. Among them, the commonly used traditional segmentation algorithms include threshold segmentation algorithms, region growing segmentation algorithms, and edge detection segmentation algorithms. Deep learning segmentation algorithms are mainly divided into two core frameworks. One is a framework for feature extraction of images based on convolutional neural network (CNN) feature encoding, and the other is an upsampling / deconvolution segmentation framework based on a fully convolutional network (FCN). The main segmentation algorithms derived from it are: Mask-R-CNN, DeepLab, PSPNet, U-net, etc.

[0004] In the process of identifying and extracting infrared spectrum equipment information, due to the complex environment of the equipment site, there are problems such as object occlusion and multiple devices being identified in the same spectrum. Infrared spectrum images are pseudo-color images, and the image segmentation model parameters obtained by directly training with pseudo-color images are difficult to promote and apply. Summary of the invention

[0005] The purpose of this application is to provide an infrared intelligent diagnosis method, device and system based on improved YOLACT++. The method is based on the improved YOLACT++ neural network, from the infrared spectrum of the target device to the entire process of fault diagnosis, by combining the advantages of YOLOX and YOLACT++ neural networks, thereby improving the efficiency and accuracy of target component identification and temperature anomaly detection.

[0006] In order to achieve the above objectives, the solution of this application is:

[0007] In a first aspect, an embodiment of the present application provides an infrared intelligent diagnosis method based on improved YOLACT++, comprising:

[0008] Acquire a grayscale image of a target device; extract a target component in the target device according to the grayscale image of the target device, the target component including a first target component and a second target component; extract a Mask of the target component by improving the YOLACT++ neural network to obtain a Mask area of ​​the target component, the Mask of the target component including a Mask of the first target component and a Mask of the second target component, the improved YOLACT++ neural network is obtained by fusing the YOLOX neural network with the YOLACT++ neural network; extract effective temperature data of the target component in the Mask area; perform fault diagnosis on the effective temperature data according to the temperature characteristics.

[0009] According to the above method of the embodiment of the present application, the following additional technical features may also be provided:

[0010] Furthermore, a grayscale image of the target device is obtained, including obtaining an infrared spectrum of the target device; extracting temperature data in the infrared spectrum and eliminating abnormal data in the temperature data; performing a LUT transformation on the temperature data after eliminating the abnormal data to obtain a grayscale image of the target device.

[0011] Furthermore, the Mask of the target component is extracted by improving the YOLACT++ neural network to obtain the Mask area of ​​the target component, including inputting the grayscale image of the target device into the improved YOLACT++ neural network, passing through 5 convolution modules, outputting the depth features of the target device, and corresponding the depth features to the grayscale image to obtain multi-scale features, and the multi-scale features include C1 to C5 layers; processing the C3 to C5 layers in the multi-scale features to construct an FPN network, and the FPN network includes P3 to P7 layers; branching and parallel processing is performed on the P3 to P7 layers in the FPN network to obtain the Mask area of ​​the target component, and the Mask area of ​​the target component includes the confidence and bounding box coordinates of the first target component and the second target component respectively.

[0012] Furthermore, the C3 to C5 layers in the multi-scale features are processed to construct an FPN network, including convolving the C5 layer in the multi-scale features to obtain the P5 layer; performing bilinear interpolation amplification on the P5 layer to obtain the P5 amplification value, convolving the C4 layer in the multi-scale features, and adding the P5 amplification value to obtain the P4 layer; performing bilinear interpolation amplification on the P4 layer to obtain the P4 amplification value, convolving the C3 layer in the multi-scale features, and adding the P4 amplification value to obtain the P3 layer; convolving the P5 layer to obtain the P6 layer; and convolving the P6 layer to obtain the P7 layer.

[0013] Furthermore, the P3 to P7 layers in the FPN network are branched and processed in parallel to obtain the Mask area of ​​the target component, including inputting the P3 layer into the Protonet module to obtain K prototype masks of the target device; inputting the P3 to P7 layers into the Prediction Head module to obtain the confidence and bounding box coordinates of the first target component; and inputting the P5 to P7 layers into the YOLOXHead module to obtain the confidence and bounding box coordinates of the second target component.

[0014] Furthermore, the Protonet module and the Prediction Head module are linearly weighted, and the Mask of the first target component is cut in the bounding box coordinates of the first target component to obtain the mask integration of the first target component; the YOLOX Head module and the Prediction Head module are linearly weighted, and the Mask of the second target component is screened in the first target component.

[0015] Furthermore, a threshold is preset, and the first target component is screened in the Mask of the second target component according to the preset threshold to obtain the Mask area of ​​the first target component and the Mask area of ​​the second target component.

[0016] In a second aspect, the embodiment of the present application provides an infrared intelligent diagnostic device based on improved YOLACT++, comprising:

[0017] An image acquisition module is configured to acquire a grayscale image of a target device and extract a target component in the target device according to the grayscale image of the target device, wherein the target component includes a first target component and a second target component;

[0018] A mask processing module is configured to extract a mask of a target component by improving a YOLACT++ neural network to obtain a mask area of ​​the target component, wherein the mask of the target component includes a mask of a first target component and a mask of a second target component, and the improved YOLACT++ neural network is obtained by fusing a YOLOX neural network with a YOLACT++ neural network;

[0019] The fault diagnosis module is configured to extract effective temperature data of the target component in the Mask area and perform fault diagnosis on the effective temperature data according to the temperature characteristics.

[0020] In the third aspect, an embodiment of the present application provides an infrared intelligent diagnosis system based on improved YOLACT++, the system includes a processor and a memory, the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the infrared intelligent diagnosis method based on improved YOLACT++ as provided in the first aspect of the embodiment of the present application.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the infrared intelligent diagnosis method based on improved YOLACT++ as in the first aspect of the embodiment of the present application.

[0022] Compared with the prior art, the infrared intelligent diagnosis method based on the improved YOLACT++ provided in the embodiment of the present application has the following beneficial technical effects:

[0023] The embodiment of the present application avoids the data conversion errors that may exist in the traditional method by directly extracting temperature data from the infrared spectrum; converting the temperature data into a grayscale image makes the temperature distribution intuitive and visible, which is convenient for subsequent processing and analysis, and provides high-quality input data for subsequent image processing and deep learning algorithms;

[0024] The embodiment of the present application can accurately identify and locate key components in the target device by improving the YOLACT++ neural network, laying the foundation for subsequent temperature data extraction and fault diagnosis, eliminating interference from non-critical areas, and making subsequent analysis more focused and effective;

[0025] The embodiment of the present application improves the YOLACT++ neural network to extract the Mask mask, and uses the powerful capabilities of the deep learning algorithm to quickly and accurately extract the Mask mask of the target component, including the confidence and bounding box coordinates; constructs multi-scale features through the FPN network to enhance the detection capability of target components of different sizes; combines the advantages of YOLOX and YOLACT++ to improve the accuracy and speed of detection;

[0026] The temperature data extracted from the Mask area in the embodiment of the present application is more accurate, reflects the actual temperature state of the target component, excludes the temperature data of the non-target area, and reduces noise interference;

[0027] The embodiment of the present application performs fault diagnosis on effective temperature data according to temperature characteristics. Fault diagnosis based on temperature characteristics can quickly identify abnormalities or potential problems. Through automated and intelligent diagnostic processes, the need for manual intervention is reduced, diagnostic efficiency and accuracy are improved, preventive maintenance and fault prediction are supported, and problems can be discovered and solved in advance, reducing equipment downtime and maintenance costs.

[0028] The embodiment of the present application constructs an FPN network through methods such as convolution and bilinear interpolation, realizes the fusion of multi-scale features, improves the detection capability of targets of different sizes, improves the efficiency and accuracy of Mask extraction through parallel processing of the Protonet module, the Prediction Head module and the YOLOX Head module, and screens the first target component in the Mask of the second target component, further improving the robustness and accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the process of an infrared intelligent diagnosis method based on improved YOLACT++ according to an embodiment of the present application is shown;

[0030] Figure 2 The structural block diagram of the infrared intelligent diagnosis device based on the improved YOLACT++ in the embodiment of the present application is shown;

[0031] Figure 3 A structural block diagram of a computer device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0033] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0034] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] The improved YOLACT++ in the embodiment of the present application is a deep learning algorithm. YOLACT++ itself is a further improvement and optimization based on YOLACT (You Only Look At Coefficients Once), which is developed based on the YOLO (You Only Look Once) series of target detection algorithms. These algorithms are important applications of deep learning in the field of computer vision, especially in target detection tasks.

[0036] Deep learning algorithms build multi-layer neural network models and use large amounts of data for training to automatically learn the feature representation of data, and then complete various complex tasks such as image classification, target detection, semantic segmentation, etc. YOLACT++, as a target detection algorithm, combines instance segmentation and target detection tasks. It can detect multiple targets in an image at the same time and give each target's precise bounding box and segmentation mask.

[0037] In YOLACT++, the FPN (Feature Pyramid Networks) structure is introduced to construct multi-scale features, and the Mask extraction process is optimized through the Protonet module and parallel processing mechanism, so that the algorithm can maintain a high detection speed while also improving the detection accuracy and segmentation effect.

[0038] Therefore, the improved YOLACT++ in the embodiment of the present application not only inherits the high efficiency and real-time performance of the YOLO series algorithms, but also further improves the performance and effect of the algorithm by introducing more deep learning technologies and optimization methods.

[0039] like Figure 1 As shown, the embodiment of the present application provides an infrared intelligent diagnosis method based on improved YOLACT++, comprising the following steps:

[0040] Step 101, obtaining a grayscale image of a target device.

[0041] The embodiment of the present application obtains an infrared spectrum of the target device, extracts temperature data in the infrared spectrum, removes abnormal data in the temperature data, performs a LUT transformation on the temperature data after removing the abnormal data, and obtains a grayscale image of the target device.

[0042] Specifically, use infrared thermal imagers and other equipment to scan the target device and obtain its infrared radiation spectrum. The infrared spectrum directly reflects the temperature distribution of the target device and provides a basis for subsequent temperature data extraction. Infrared detection does not require direct contact with the target device, reducing the impact on the equipment and the risk of damage during the measurement process.

[0043] Extract temperature data from the infrared spectrum, parse the temperature value corresponding to each pixel from the infrared spectrum to form a temperature data set, and convert the infrared spectrum into specific temperature data to facilitate quantitative analysis and processing, providing the necessary data basis for subsequent data elimination, transformation and fault diagnosis.

[0044] Eliminate abnormal data in the temperature data and perform outlier detection on the extracted temperature data, such as identifying and eliminating abnormal data through threshold judgment, statistical methods, etc. Eliminating abnormal data can reduce data noise, improve the accuracy and reliability of subsequent analysis, and make the entire processing flow more robust and able to cope with complex and changing measurement environments.

[0045] The temperature data after removing abnormal data is transformed by LUT to obtain the grayscale image of the target device. The temperature data after removing abnormal values ​​is mapped and transformed using a lookup table (LUT). The temperature value is mapped to a grayscale value range (usually 0-255) to generate a grayscale image. The temperature data is converted into a grayscale image to make the temperature distribution intuitive and visible, which is convenient for subsequent processing and analysis. The temperature data is standardized into grayscale values ​​through LUT transformation, which provides a unified input format for subsequent image processing and deep learning algorithms. The parameters of the LUT transformation can be adjusted as needed to enhance the contrast of the image and highlight areas with large temperature differences.

[0046] Step 102: extracting a target component in the target device according to the grayscale image of the target device, where the target component includes a first target component and a second target component.

[0047] In the process of extracting a target component according to a grayscale image of a target device, the embodiment of the present application includes a first target component and a second target component, which is implemented by image processing technology and possible deep learning algorithms.

[0048] Specifically, the grayscale image is preprocessed and necessary preprocessing is performed on the grayscale image, such as filtering, denoising, contrast enhancement, etc., to improve the accuracy and efficiency of subsequent processing, improve image quality, reduce noise and interference, and provide better input for subsequent target component extraction.

[0049] Target component detection uses image processing algorithms such as edge detection, threshold segmentation, morphological operations or deep learning algorithms such as convolutional neural networks (CNNs) and target detection models to identify target components in grayscale images. The target components are distinguished based on their features such as shape, size, position, texture, etc., and the first target component and the second target component are extracted respectively. Through advanced image processing technology and deep learning algorithms, the target components can be accurately identified and located, reducing the possibility of misidentification and missed identification. The automated processing reduces the need for manual intervention and improves processing efficiency and accuracy.

[0050] The extracted target components are optimized, such as boundary smoothing and small debris removal, to improve the accuracy of subsequent analysis and verify the correctness of the extraction results. This can be done by comparing with known data, manual inspection, etc. Through optimization, the accuracy of target component extraction can be further improved, providing a more reliable data basis for subsequent temperature data extraction and fault diagnosis. Through verification steps, the correctness and reliability of the extraction results can be ensured, providing confidence for subsequent analysis and application.

[0051] If a deep learning algorithm is used for target component detection, feature extraction and classification capabilities can be used to achieve more accurate and robust target component recognition. When processing grayscale images containing multiple target components, it is necessary to ensure that the algorithm can simultaneously recognize and distinguish different target components, such as the first target component and the second target component.

[0052] In practical applications, grayscale images may be affected by various factors such as illumination changes, occlusion, and noise, which makes it more difficult to extract target components. Therefore, in practical applications, it is necessary to optimize and adjust the algorithm according to specific circumstances.

[0053] Step 103, extracting the Mask of the target component by improving the YOLACT++ neural network to obtain the Mask area of ​​the target component, the Mask of the target component includes the Mask of the first target component and the Mask of the second target component, and the improved YOLACT++ neural network is obtained by fusing the YOLOX neural network with the YOLACT++ neural network.

[0054] The fusion of the YOLOX neural network and the YOLACT++ neural network to obtain an improved YOLACT++ neural network involves multiple considerations and optimizations. Among them, YOLOX, as a variant of the YOLO series, is known for its high performance and real-time performance, especially in target detection. YOLACT++ is a model focused on real-time instance segmentation, which achieves fast and relatively accurate instance segmentation through parallel processing. The fusion of the two aims to combine the advantages of YOLOX in detection and the capabilities of YOLACT++ in segmentation to improve the performance of instance segmentation.

[0055] Specifically, select Darknet53 or a lighter network used in YOLOX as the backbone network to extract image features. Keep the FPN structure in YOLACT++ to aggregate features at different scales to detect objects of different sizes. Learn from strategies such as Decoupled Head and SimOTA in YOLOX to optimize the detection module and improve the accuracy and efficiency of target detection. Keep the Protonet branch and Prediction Head branch in YOLACT++ to generate prototype masks and mask coefficients. Combine data enhancement methods such as Mosaic and Mixup in YOLOX to increase the diversity of training data and improve the generalization ability of the model. Use the Cosine Annealing with Warmup learning rate scheduling strategy in YOLOX, as well as possible automatic data enhancement strategies (Auto-Augment) to optimize the training process. Combine the loss function design in YOLOX and YOLACT++ to ensure that both the target can be accurately detected and the instance can be accurately segmented. Select a suitable optimizer (such as SGD, Adam, etc.) and adjust hyperparameters such as learning rate to accelerate model convergence and improve performance. Use the NMS optimization strategy (such as Fast-NMS) in YOLOX or YOLACT++ to reduce the overlap of detection boxes and improve detection accuracy. Use standard instance segmentation evaluation indicators (such as mAP, FPS, etc.) to evaluate the performance of the fused model.

[0056] The improved YOLACT++ neural network of the embodiment of the present application integrates YOLOX and YOLACT++, wherein YOLOX is a high-performance target detection network that excels in both speed and accuracy, and YOLACT++ is an instance segmentation network that can provide high-quality Mask masks at real-time speed.

[0057] By combining the head of YOLOX with the Protonet and Prediction Head of YOLACT++, an improved neural network can be built that can perform efficient target detection and generate accurate Mask masks.

[0058] Furthermore, the grayscale image of the target device is input into the improved YOLACT++ neural network, and after 5 convolution modules (such as ResNet, etc.), the depth features of the target device are output, and the depth features are mapped to the grayscale image to obtain multi-scale features. The multi-scale features include C1 to C5 layers, and each layer captures information of different scales.

[0059] The C3 to C5 layers in the multi-scale features are processed to construct an FPN network, which includes layers P3 to P7; the C5 layer in the multi-scale features is convolved to obtain the P5 layer; the P5 layer is bilinearly interpolated and amplified to obtain the P5 amplified value, the C4 layer in the multi-scale features is convolved and added with the P5 amplified value to obtain the P4 layer; the P4 layer is bilinearly interpolated and amplified to obtain the P4 amplified value, the C3 layer in the multi-scale features is convolved and added with the P4 amplified value to obtain the P3 layer; the P5 layer is convolved to obtain the P6 layer; the P6 layer is convolved to obtain the P7 layer, so as to improve the network's detection ability for targets of different scales.

[0060] The P3 to P7 layers in the FPN network are branched and processed in parallel to obtain the Mask area of ​​the target component, including inputting the P3 layer into the Protonet module to obtain K prototype masks of the target device, which are the basis of the Mask; inputting the P3 to P7 layers into the Prediction Head module to obtain the confidence and bounding box coordinates of the first target component; inputting the P5 to P7 layers into the YOLOX Head module to obtain the confidence and bounding box coordinates of the second target component. At the same time, it may be necessary to further exclude the part overlapping with the first target component.

[0061] It should be noted that regarding the processing of the C1 and C2 layers, the design of FPN usually does not directly involve these two layers, because they have a high resolution in the feature pyramid and relatively less semantic information, and are not suitable for direct subsequent processing in the target detection task.

[0062] The Mask area of ​​the target component includes the confidence and bounding box coordinates of the first target component and the second target component respectively. The Protonet module and the Prediction Head module are linearly weighted, and the Mask of the first target component is cut in the bounding box coordinates of the first target component to obtain the mask integration of the first target component; the YOLOXHead module and the Prediction Head module are linearly weighted, and the first target component is screened in the Mask of the second target component according to a preset threshold to obtain the Mask area of ​​the first target component and the Mask area of ​​the second target component.

[0063] It should be noted that in practical applications, the improved YOLACT++ neural network needs to be trained to learn how to accurately detect and segment target components from grayscale images. The parameters in the network, such as convolution kernel size, step size, activation function, loss function, etc., need to be adjusted and optimized according to specific tasks and data sets. During and after training, appropriate evaluation indicators such as mAP, IoU, etc. need to be used to evaluate the performance of the network. Considering the needs of real-time applications, the design and optimization of the network should ensure that the processing speed is maximized while maintaining high accuracy.

[0064] Step 104, extracting effective temperature data of the target component in the Mask area; and performing fault diagnosis on the effective temperature data according to the temperature characteristics.

[0065] After extracting the Mask areas of the target components, the embodiment of the present application further processes these areas to obtain valid temperature data of the target components, and performs fault diagnosis based on the temperature data.

[0066] Specifically, extract the effective temperature data in the mask area, and correspond the grayscale image to the temperature data. First, ensure a temperature data map corresponding to the grayscale image of the target device, usually a temperature matrix with the same resolution as the grayscale image. Each pixel value in the temperature data map represents the temperature at the corresponding position.

[0067] The extracted Mask, whether it is the first target component or the second target component, is applied to the temperature data map through pixel-by-pixel multiplication operations or equivalent logical operations. If the Mask is binary, the temperature data in the non-target component area is set to an invalid value, such as NaN, 0, or a specific negative value, depending on the requirements of subsequent processing.

[0068] Extract the effective temperature data in the target component area from the temperature data graph after mask processing, including statistical information such as the average temperature, maximum temperature, minimum temperature, and temperature distribution histogram in the calculation area.

[0069] Perform fault diagnosis based on temperature characteristics and set temperature thresholds. Set reasonable temperature thresholds based on the normal operating temperature range of the target component. These thresholds can be fixed, such as based on design specifications or empirical data, or dynamic, such as based on real-time operating conditions or historical data.

[0070] Compare the extracted valid temperature data with the set threshold to identify temperature values ​​that are outside the normal range, which may indicate potential faults or abnormal working conditions. Perform fault diagnosis analysis based on the type, location, duration and other information of the temperature anomaly. Further, it may be necessary to combine other sensor data, equipment logs, historical fault records and other information to comprehensively determine the cause of the fault. Organize the results of the fault diagnosis into a report, including fault description, location, possible causes, recommended repair measures, etc. The report can be provided to maintenance personnel or automation systems to guide subsequent maintenance or adjustment work.

[0071] Based on the actual fault diagnosis results and maintenance effects, the temperature threshold, diagnostic algorithm and report template are continuously optimized. Through continuous learning and feedback, the accuracy and efficiency of fault diagnosis are improved.

[0072] It should be noted that temperature data is only one aspect of fault diagnosis. In actual applications, it may be necessary to combine multiple sensor data and intelligent algorithms for comprehensive analysis and judgment. In addition, for complex equipment or systems, it may also be necessary to consider the correlation and mutual influence between temperature data and other physical quantities, such as pressure, flow, vibration, etc.

[0073] like Figure 2 As shown, the embodiment of the present application provides an infrared intelligent diagnosis device based on improved YOLACT++, including an image acquisition module 201, a mask processing module 202 and a fault diagnosis module 203, wherein:

[0074] The image acquisition module 201 is configured to acquire a grayscale image of a target device and extract a target component in the target device according to the grayscale image of the target device, wherein the target component includes a first target component and a second target component;

[0075] The mask processing module 202 is configured to extract the mask of the target component by improving the YOLACT++ neural network to obtain the mask area of ​​the target component, wherein the mask of the target component includes the mask of the first target component and the mask of the second target component, and the improved YOLACT++ neural network is obtained by fusing the YOLOX neural network with the YOLACT++ neural network;

[0076] The fault diagnosis module 203 is configured to extract effective temperature data of the target component in the Mask area and perform fault diagnosis on the effective temperature data according to the temperature characteristics.

[0077] The infrared intelligent diagnostic device based on improved YOLACT++ in the embodiment of the present application can be a computer device, or a component in a computer device, such as an integrated circuit or a chip. The computer device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the computer device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted computer device, a mobile Internet device (Mobile Internet Device, MID), an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), etc., which is not specifically limited in the embodiment of the present application.

[0078] The infrared intelligent diagnostic device based on the improved YOLACT++ provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the infrared intelligent diagnosis method embodiment based on the improved YOLACT++ will not be repeated here.

[0079] The present application also provides a computer device, such as Figure 3 As shown, the computer device includes a processor 301 and a memory 302. The memory 302 stores programs or instructions that can be executed on the processor 301. When the program or instructions are executed by the processor 301, the various steps of the above-mentioned infrared intelligent diagnosis method based on the improved YOLACT++ are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0080] It should be noted that the computer device in the embodiment of the present application includes the above-mentioned mobile computer device and non-mobile computer device.

[0081] The memory 302 can be used to store software programs and various data. The memory 302 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 302 may include a volatile memory or a non-volatile memory, or the memory 302 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 302 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0082] The processor 301 may include one or more processing units; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 301.

[0083] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned infrared intelligent diagnosis method embodiment based on improved YOLACT++ are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0084] An embodiment of the present application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned infrared intelligent diagnosis method embodiment based on improved YOLACT++, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0085] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0086] An embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned infrared intelligent diagnosis method embodiment based on improved YOLACT++, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0087] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0088] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An infrared intelligent diagnosis method based on improved YOLACT++, characterized in that: The method comprises: Get a grayscale image of the target device; Extracting a target component in the target device according to the grayscale image of the target device, wherein the target component includes a first target component and a second target component; Extracting the Mask of the target component by improving the YOLACT++ neural network to obtain the Mask area of ​​the target component, wherein the Mask of the target component includes the Mask of the first target component and the Mask of the second target component, and the improved YOLACT++ neural network is obtained by fusing the YOLOX neural network with the YOLACT++ neural network; Extracting effective temperature data of the target component in the Mask area; Fault diagnosis is performed on the effective temperature data according to the temperature characteristics.

2. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 1, characterized in that: The step of obtaining a grayscale image of a target device includes: Acquire an infrared spectrum of the target device; Extracting temperature data from the infrared spectrum and removing abnormal data from the temperature data; The temperature data after the abnormal data is removed is subjected to LUT transformation to obtain a grayscale image of the target device.

3. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 1, characterized in that: The Mask mask of the target component is extracted by improving the YOLACT++ neural network to obtain the Mask mask area of ​​the target component, including: Input the grayscale image of the target device into the improved YOLACT++ neural network, pass through 5 convolution modules, output the deep features of the target device, correspond the deep features to the grayscale image, and obtain multi-scale features, wherein the multi-scale features include C1 to C5 layers; Processing C3 to C5 layers in the multi-scale features to construct an FPN network, wherein the FPN network includes P3 to P7 layers; The P3 to P7 layers in the FPN network are processed in parallel by branches to obtain a Mask area of ​​the target component, where the Mask area of ​​the target component includes respective confidence levels and bounding box coordinates of the first target component and the second target component.

4. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 3, characterized in that: The processing of the C3 to C5 layers in the multi-scale features to construct an FPN network includes: Convolving the C5 layer in the multi-scale features to obtain the P5 layer; Perform bilinear interpolation amplification on the P5 layer to obtain a P5 amplification value, perform convolution on the C4 layer in the multi-scale feature, and add the convolution with the P5 amplification value to obtain the P4 layer; Perform bilinear interpolation amplification on the P4 layer to obtain a P4 amplification value, perform convolution on the C3 layer in the multi-scale feature, and add the convolution with the P4 amplification value to obtain a P3 layer; The P5 layer is convolved to obtain the P6 layer; the P6 layer is convolved to obtain the P7 layer.

5. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 3, characterized in that: The branching and parallel processing of the P3 to P7 layers in the FPN network to obtain the Mask area of ​​the target component includes: Input the P3 layer into the Protonet module to obtain K prototype masks of the target device; Input the P3 to P7 layers into the Prediction Head module to obtain the confidence and bounding box coordinates of the first target component; Input layers P5 to P7 into the YOLOX Head module to obtain the confidence and bounding box coordinates of the second target component.

6. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 5, characterized in that: The method comprises: Linearly weighting the Protonet module and the Prediction Head module, cutting the Mask of the first target component in the bounding box coordinates of the first target component, and obtaining the mask integration of the first target component; The YOLOX Head module and the Prediction Head module are linearly weighted, and the first target component is screened in the Mask mask of the second target component.

7. The infrared intelligent diagnosis method based on improved YOLACT++ as claimed in claim 6, characterized in that: The method comprises: A preset threshold is set, and the first target component is screened in the Mask of the second target component according to the preset threshold to obtain the Mask area of ​​the first target component and the Mask area of ​​the second target component.

8. An infrared intelligent diagnostic device based on improved YOLACT++, characterized in that: The device comprises: An image acquisition module is configured to acquire a grayscale image of a target device and extract a target component in the target device according to the grayscale image of the target device, wherein the target component includes a first target component and a second target component; A mask processing module is configured to extract the Mask mask of the target component by improving the YOLACT++ neural network to obtain the Mask mask area of ​​the target component, wherein the Mask mask of the target component includes the Mask mask of the first target component and the Mask mask of the second target component, and the improved YOLACT++ neural network is obtained by fusing the YOLOX neural network with the YOLACT++ neural network; The fault diagnosis module is configured to extract effective temperature data of the target component in the Mask area and perform fault diagnosis on the effective temperature data according to temperature characteristics.

9. An infrared intelligent diagnosis system based on improved YOLACT++, the system comprising a processor and a memory, wherein a computer program is stored in the memory, characterized in that: The computer program is loaded and executed by the processor to implement the infrared intelligent diagnosis method based on improved YOLACT++ as described in any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the storage medium, characterized in that: When the computer program is executed by a processor, it is used to implement the infrared intelligent diagnosis method based on improved YOLACT++ as described in any one of claims 1 to 7.

Images