Power transmission line infrared image instance segmentation method and device, electronic equipment and storage medium
Through the environment-aware segmentation model, selecting appropriate instance segmentation sub-models according to the environment category, the problem of unstable segmentation effect in the prior art under different environments is solved, and higher segmentation accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411650353.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-04-04
AI Technical Summary
The existing infrared image instance segmentation technology of power transmission lines is unstable in different environments, especially in complex backgrounds such as cities, farmlands and forests, where missed or missed segmentation is prone to occur.
Through the environment-aware segmentation model, an appropriate instance segmentation sub-model is selected according to the environmental category of infrared images of the transmission line to generate an instance mask. The model includes an environment classification submodel, a gated convolutional submodel and an instance segmentation submodel group. Through the combination of these submodels, the confidence that the image belongs to each environmental category is determined, and the most suitable segmentation model is selected for segmentation.
The accuracy and robustness of instance segmentation of infrared images of transmission lines in different environments is significantly improved, the performance of the unified model in complex backgrounds is avoided, and the accuracy and efficiency of segmentation is improved.
Smart Images
Figure CN119762768A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, electronic equipment and storage medium for instance segmentation of infrared images of power transmission lines. Background Art
[0002] Infrared image instance segmentation technology for transmission lines plays an important role in power inspection. Infrared imaging technology can work stably in weak light and bad weather conditions by sensing the temperature difference between the background and the target, and has been widely used in transmission line inspection. Infrared images can provide inspectors with real-time temperature distribution information and help to promptly identify abnormal conditions of electrical equipment, such as overheating or damage. However, infrared images often contain complex backgrounds and noise, and the shapes and sizes of target objects vary greatly. Therefore, instance segmentation technology is needed to accurately extract the contour information of electrical equipment from images.
[0003] Most existing instance segmentation techniques rely on deep learning models, which can automatically learn multi-level features in images and thus improve the segmentation effect in complex scenes. However, due to the complexity and diversity of actual environments, existing segmentation methods usually rely on unified models or fixed strategies for instance segmentation. These methods input all images into the same segmentation model for processing, but the differences in the environment, such as the differences in backgrounds such as cities, farmlands, and forests, will significantly affect the feature distribution and structure of the image. Since the unified model cannot be adjusted for these environmental differences, the segmentation effect of the model for certain environments is unstable. For example, a unified segmentation method may achieve good segmentation results in an urban environment, but in a farmland or forest environment, due to the interference of vegetation or other natural factors, the segmentation results may be affected, and even mis-segmentation or missed segmentation may occur. This problem of poor environmental adaptability seriously affects the accuracy of segmentation. Summary of the invention
[0004] The embodiment of the present invention provides a method, device, electronic device and storage medium for instance segmentation of transmission line infrared images. By implementing the present invention, a suitable instance segmentation model can be selected according to the environment category of the transmission line infrared image, thereby improving the accuracy and robustness of instance segmentation of the transmission line infrared image in different environments.
[0005] An embodiment of the present invention provides an instance segmentation method for an infrared image of a power transmission line, comprising:
[0006] Acquire an infrared image of the transmission line to be segmented;
[0007] The infrared image to be segmented is input into the environment-aware segmentation model so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, and the environment category with the highest confidence is taken as the target environment category. The infrared image to be segmented is segmented according to the instance segmentation sub-model corresponding to the target environment category, and an instance mask of the infrared image to be segmented is generated; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
[0008] Furthermore, the confidence that the infrared image to be segmented belongs to each environment category is determined by:
[0009] Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category;
[0010] According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
[0011] Furthermore, the environment classification sub-model is trained in the following way:
[0012] Acquire a plurality of first infrared images of the power transmission line and corresponding environmental labels; wherein the environmental label is the environmental category of the first infrared image;
[0013] Repeat the training process of the environment classification sub-model until a preset first execution number is reached, and the training process of the environment classification sub-model includes:
[0014] Randomly dividing the first infrared image into a plurality of batches of first training samples according to a preset first quantity;
[0015] Input the batches of first training samples into the current environment classification sub-model in sequence, so that when the current environment classification sub-model receives each batch of first training samples, it outputs the probability value of the first training sample belonging to each environment category through the current environment classification sub-model according to the input first training sample; calculate the first loss function value through the cross entropy loss according to the probability value and the corresponding environment label; and update the current environment classification sub-model according to the first loss function value using an optimizer;
[0016] When the preset first execution times are reached, the finally updated environment classification sub-model is set as the final environment classification sub-model.
[0017] Furthermore, the gated convolution sub-model is trained in the following way:
[0018] Obtain probability vectors and corresponding confidence labels of several second infrared images of the transmission line; wherein the probability vector is composed of probability values of the infrared image belonging to each environmental category; the probability values are generated by the environmental classification sub-model according to the infrared image; and the confidence label is a one-hot encoding vector of the environmental category corresponding to the second infrared image;
[0019] Repeat the training process of the gated convolution sub-model until a preset second number of executions is reached, wherein the training process of the gated convolution sub-model includes:
[0020] Randomly dividing the probability vector into a plurality of batches of second training samples according to a preset second number;
[0021] Input the batches of second training samples into the current gated convolution sub-model in sequence, so that when the current gated convolution sub-model receives each batch of second training samples, it outputs a confidence vector corresponding to the second training sample through the current gated convolution sub-model according to the input second training sample; wherein the confidence vector includes the confidence of each environment category of the second infrared image; calculate the second loss function value through the cross entropy loss according to the confidence vector and the corresponding confidence label; and update the current gated convolution sub-model according to the second loss function value using an optimizer;
[0022] When the preset second execution number is reached, the finally updated gated convolution sub-model is set as the final gated convolution sub-model.
[0023] Furthermore, the instance segmentation sub-model is trained in the following way:
[0024] Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images;
[0025] Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising:
[0026] randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number;
[0027] Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer;
[0028] When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
[0029] Based on the above method embodiment, the present invention provides a corresponding device embodiment.
[0030] An embodiment of the present invention provides an instance segmentation device for infrared images of power transmission lines, comprising: an image acquisition module and an instance segmentation module;
[0031] The image acquisition module is used to acquire the infrared image of the power transmission line to be segmented;
[0032] The instance segmentation module is used to input the infrared image to be segmented into the environment-aware segmentation model so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, takes the environment category with the highest confidence as the target environment category, and segments the infrared image to be segmented according to the instance segmentation sub-model corresponding to the target environment category to generate an instance mask of the infrared image to be segmented; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
[0033] Furthermore, the instance segmentation module determines the confidence that the infrared image to be segmented belongs to each environment category in the following way:
[0034] Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category;
[0035] According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
[0036] Furthermore, the instance segmentation sub-model is trained in the following way:
[0037] Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images;
[0038] Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising:
[0039] randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number;
[0040] Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer;
[0041] When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
[0042] Based on the above method embodiment, the present invention provides a corresponding electronic device embodiment.
[0043] An embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the instance segmentation method of the infrared image of the transmission line described in any one of the above method embodiments can be implemented.
[0044] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.
[0045] An embodiment of the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for instance segmentation of infrared images of power transmission lines described in any one of the above method embodiments can be implemented.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] The embodiment of the present invention provides an instance segmentation method, device, electronic device and storage medium for infrared images of power transmission lines. The method performs instance segmentation of infrared images of power transmission lines through an environment perception segmentation model, generates a probability value that the infrared image to be segmented belongs to each environment category in a built-in environment classification model; calibrates the probability value of each environment category in a built-in gated convolution model to generate the confidence of each environment category; selects the instance segmentation model corresponding to the environment category in the built-in instance segmentation model group according to the confidence, and segments the infrared image to be segmented in the instance segmentation model corresponding to the environment category to generate an instance mask of the infrared image to be segmented. Through the combination of the environment classification model, the gated convolution model and the instance segmentation model group, after the image is input, the probability value of each environment category is generated, and the gated convolution model is used to calibrate it to generate an accurate confidence. According to these confidences, the most suitable instance segmentation model is selected, so as to flexibly adjust according to the characteristics of different environments, significantly improving the segmentation effect of the model in different environments, not only avoiding the performance degradation of the unified model under complex backgrounds, but also improving the accuracy and robustness of segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 The present invention is a flowchart of an example segmentation method of an infrared image of a power transmission line provided by an embodiment of the present invention.
[0049] Figure 2 The diagram is a schematic structural diagram of an example segmentation device for infrared images of a power transmission line provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] like Figure 1 As shown, an embodiment of the present invention provides an instance segmentation method for an infrared image of a power transmission line, which at least includes the following steps:
[0052] Step S1, obtaining an infrared image of a power transmission line to be segmented;
[0053] It is understandable that the infrared images of the transmission lines to be segmented are usually taken using infrared cameras. Infrared images can provide high-quality image data and provide accurate input for the subsequent environmental perception segmentation model. By collecting infrared images, the thermal imaging features of the transmission lines can be accurately captured and the status and location of electrical equipment can be identified.
[0054] Step S2, inputting the infrared image to be segmented into the environment-aware segmentation model, so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, taking the environment category with the highest confidence as the target environment category, and segmenting the infrared image to be segmented according to the instance segmentation sub-model corresponding to the target environment category, and generating an instance mask of the infrared image to be segmented; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
[0055] Specifically, the confidence that the infrared image to be segmented belongs to each environment category is determined in the following way:
[0056] Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category;
[0057] According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
[0058] Optionally, train the environment classification sub-model by:
[0059] Acquire a plurality of first infrared images of the power transmission line and corresponding environmental labels; wherein the environmental label is the environmental category of the first infrared image;
[0060] Repeat the training process of the environment classification sub-model until a preset first execution number is reached, and the training process of the environment classification sub-model includes:
[0061] Randomly dividing the first infrared image into a plurality of batches of first training samples according to a preset first quantity;
[0062] Input the batches of first training samples into the current environment classification sub-model in sequence, so that when the current environment classification sub-model receives each batch of first training samples, it outputs the probability value of the first training sample belonging to each environment category through the current environment classification sub-model according to the input first training sample; calculate the first loss function value through the cross entropy loss according to the probability value and the corresponding environment label; and update the current environment classification sub-model according to the first loss function value using an optimizer;
[0063] When the preset first execution times are reached, the finally updated environment classification sub-model is set as the final environment classification sub-model.
[0064] It should be noted that when training the environment classification sub-model, the first infrared image of the transmission line is input into the encoder and full connector of the environment classification sub-model, and feature extraction and environment category prediction are performed respectively, and multi-class cross entropy loss is set to achieve parameter optimization of the environment classification model under supervised conditions; among them, the environment classification sub-model is an existing model, which can be selected from models such as AlexNet, VGG, ResNet and DenseNet, but the output dimension of the last layer of the full connector is consistent with the number of environment categories.
[0065] In a specific embodiment, the infrared image contains four environmental categories, namely, city, farmland, forest and sky. For an infrared image, the environmental classification sub-model will analyze the image, extract the features therein, and then predict the probability that the image belongs to each environmental category. After analysis by the environmental classification sub-model, the probability value of the infrared image to be segmented belonging to each environmental category may be: [0.05, 0.2, 0.15, 0.6], indicating that the probability that the image belongs to the urban environment is 0.05, the probability that the image belongs to the farmland environment is 0.2, the probability that the image belongs to the forest environment is 0.15, and the probability that the image belongs to the sky environment is 0.6; from the probability value of the infrared image belonging to each environmental category, it can be seen that this image is most likely to belong to the sky environment with a probability of 0.6. These probability values will provide a basis for the subsequent instance segmentation sub-model and help select a suitable segmentation strategy.
[0066] Optionally, train the gated convolutional sub-model as follows:
[0067] Obtain probability vectors and corresponding confidence labels of several second infrared images of the transmission line; wherein the probability vector is composed of probability values of the infrared image belonging to each environmental category; the probability values are generated by the environmental classification sub-model according to the infrared image; and the confidence label is a one-hot encoding vector of the environmental category corresponding to the second infrared image;
[0068] Repeat the training process of the gated convolution sub-model until a preset second number of executions is reached, wherein the training process of the gated convolution sub-model includes:
[0069] Randomly dividing the probability vector into a plurality of batches of second training samples according to a preset second number;
[0070] Input the batches of second training samples into the current gated convolution sub-model in sequence, so that when the current gated convolution sub-model receives each batch of second training samples, it outputs a confidence vector corresponding to the second training sample through the current gated convolution sub-model according to the input second training sample; wherein the confidence vector includes the confidence of each environment category of the second infrared image; calculate the second loss function value through the cross entropy loss according to the confidence vector and the corresponding confidence label; and update the current gated convolution sub-model according to the second loss function value using an optimizer;
[0071] When the preset second execution number is reached, the finally updated gated convolution sub-model is set as the final gated convolution sub-model.
[0072] It should be noted that the gated convolution sub-model is a neural network model, which includes a convolution layer and a sigmoid activation function, wherein the sigmoid activation function is responsible for mapping the output of the convolution layer to the confidence of each environment category.
[0073] In one embodiment, the infrared image contains four environmental categories, namely, city, farmland, forest and sky. For an infrared image, if the image belongs to the "farmland" environment, the confidence label (i.e., one-hot encoding vector) will be: [0,1,0,0]; where the position of 1 indicates that the image belongs to the "farmland" category, and the other positions are 0, indicating that the image does not belong to other environmental categories.
[0074] In another embodiment, the probability vector of the infrared image is: [0.05, 0.2, 0.15, 0.6], indicating that the probability of the image belonging to the city is 0.05, the probability of the image belonging to the farmland is 0.2, the probability of the image belonging to the forest is 0.15, and the probability of the image belonging to the sky is 0.6; these probability values reflect the prediction results of the model for different environmental categories, but they may not clearly distinguish which categories are most likely and which categories are least likely. Therefore, it is necessary to calibrate through the gated convolution sub-model to adjust these probability values so that they are more decisive, that is, to make some probability values closer to 0 or closer to 1. The calibrated probability values may become: [0.01, 0.05, 0.05, 0.89]. In this calibrated example: the confidence of the city drops to 0.01, indicating that the image is almost impossible to belong to the urban environment; the confidence of the farmland drops to 0.05, indicating that the image is almost not in the farmland environment; the confidence of the forest drops to 0.05, which is also almost not in the forest environment. The confidence score for the sky increases to 0.89, indicating that the image is almost certainly a sky environment.
[0075] Through calibration, the gated convolution sub-model enhances the confidence of the sky environment category to be close to 1, while the confidence of other environment categories is reduced to close to 0. In this way, the model's judgment on each environment category becomes clearer, and it can more effectively select the appropriate instance segmentation sub-model, thereby improving the accuracy of the overall segmentation.
[0076] Optionally, train the instance segmentation sub-model as follows:
[0077] Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images;
[0078] Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising:
[0079] randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number;
[0080] Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer;
[0081] When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
[0082] It should be noted that when training the instance segmentation sub-model, the third infrared image of the transmission line is input into the encoder and decoder of the instance segmentation sub-model, and feature extraction and instance mask prediction are performed respectively, and a multi-class cross entropy loss is set to achieve parameter optimization of the instance segmentation sub-model under supervised conditions; among them, the instance segmentation sub-model is an existing model, which can be selected as Mask RCNN, Mask2Former, QueryInst and SOLOV2 models, but the output dimension of the last layer of the decoder is consistent with the number of target categories.
[0083] It can be understood that by determining the confidence that the infrared image to be segmented belongs to each environmental category through the environmental perception segmentation model, the environmental features of the image can be quickly identified, and the confidence of each environmental category can be calculated to ensure that the model can accurately classify the environment. Then, the model selects the environmental category with the highest confidence as the target environmental category, and further performs segmentation operations based on the instance segmentation sub-model corresponding to the category. This strategy of specifically selecting instance segmentation sub-models not only avoids the problem of unstable segmentation effects that may be caused by the unified use of a single segmentation model, but also selects the most suitable instance segmentation strategy based on the feature differences of different environments, thereby improving the accuracy and efficiency of segmentation.
[0084] Based on the above method embodiment, the present invention provides a corresponding device embodiment.
[0085] like Figure 2 As shown, an embodiment of the present invention provides an instance segmentation device for infrared images of power transmission lines, comprising: an image acquisition module 101 and an instance segmentation module 102;
[0086] The image acquisition module 101 is used to acquire an infrared image of the power transmission line to be segmented;
[0087] The instance segmentation module 102 is used to input the infrared image to be segmented into the environment-aware segmentation model so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, takes the environment category with the highest confidence as the target environment category, and segments the infrared image to be segmented according to the instance segmentation sub-model corresponding to the target environment category, and generates an instance mask of the infrared image to be segmented; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
[0088] In an optional embodiment, the instance segmentation module determines the confidence that the infrared image to be segmented belongs to each environment category in the following manner:
[0089] Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category;
[0090] According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
[0091] In a preferred embodiment, the instance segmentation sub-model is trained in the following manner:
[0092] Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images;
[0093] Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising:
[0094] randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number;
[0095] Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer;
[0096] When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
[0097] It should be noted that the embodiments of the device described above correspond to the above embodiments of the present invention, and can implement any of the methods described above in the present invention. In addition, the embodiments of the above-mentioned device are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without paying any creative work.
[0098] Based on the above method embodiment of the present invention, a corresponding electronic device embodiment is provided.
[0099] An embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the instance segmentation method of infrared images of power transmission lines described in any one of the present invention is implemented, or, when the processor executes the computer program, the functions of the modules in the above-mentioned device embodiments are implemented.
[0100] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the terminal device.
[0101] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.
[0102] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.
[0103] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0104] Based on the above method embodiment, the present invention provides a storage medium embodiment;
[0105] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned instance segmentation methods of infrared images of power transmission lines of the present invention.
[0106] The storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0107] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0108] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for instance segmentation of infrared images of power transmission lines, characterized in that: include: Acquire an infrared image of the transmission line to be segmented; The infrared image to be segmented is input into the environment-aware segmentation model so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, and the environment category with the highest confidence is taken as the target environment category. The infrared image to be segmented is segmented according to the instance segmentation sub-model corresponding to the target environment category, and an instance mask of the infrared image to be segmented is generated; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
2. The method for instance segmentation of a power transmission line infrared image according to claim 1, characterized in that: The confidence that the infrared image to be segmented belongs to each environment category is determined by: Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category; According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
3. The instance segmentation method of a power transmission line infrared image according to claim 2, characterized in that: The environment classification sub-model is trained in the following way: Acquire a plurality of first infrared images of the power transmission line and corresponding environmental labels; wherein the environmental label is the environmental category of the first infrared image; Repeat the training process of the environment classification sub-model until a preset first execution number is reached, and the training process of the environment classification sub-model includes: Randomly dividing the first infrared image into a plurality of batches of first training samples according to a preset first quantity; Input the batches of first training samples into the current environment classification sub-model in sequence, so that when the current environment classification sub-model receives each batch of first training samples, it outputs the probability value of the first training sample belonging to each environment category through the current environment classification sub-model according to the input first training sample; calculate the first loss function value through the cross entropy loss according to the probability value and the corresponding environment label; and update the current environment classification sub-model according to the first loss function value using an optimizer; When the preset first execution times are reached, the finally updated environment classification sub-model is set as the final environment classification sub-model.
4. The method for instance segmentation of a power transmission line infrared image according to claim 2, characterized in that: The gated convolutional sub-model is trained as follows: Obtain probability vectors and corresponding confidence labels of several second infrared images of the transmission line; wherein the probability vector is composed of probability values of the infrared image belonging to each environmental category; the probability values are generated by the environmental classification sub-model according to the infrared image; and the confidence label is a one-hot encoding vector of the environmental category corresponding to the second infrared image; Repeat the training process of the gated convolution sub-model until a preset second number of executions is reached, wherein the training process of the gated convolution sub-model includes: Randomly dividing the probability vector into a plurality of batches of second training samples according to a preset second number; Input the batches of second training samples into the current gated convolution sub-model in sequence, so that when the current gated convolution sub-model receives each batch of second training samples, it outputs a confidence vector corresponding to the second training sample through the current gated convolution sub-model according to the input second training sample; wherein the confidence vector includes the confidence of each environment category of the second infrared image; calculate the second loss function value through the cross entropy loss according to the confidence vector and the corresponding confidence label; and update the current gated convolution sub-model according to the second loss function value using an optimizer; When the preset second execution number is reached, the finally updated gated convolution sub-model is set as the final gated convolution sub-model.
5. The method for instance segmentation of a power transmission line infrared image according to claim 1, characterized in that: The instance segmentation sub-model is trained as follows: Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images; Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising: randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number; Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer; When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
6. An instance segmentation device for infrared images of power transmission lines, characterized in that: include: Image acquisition module and instance segmentation module; The image acquisition module is used to acquire the infrared image of the power transmission line to be segmented; The instance segmentation module is used to input the infrared image to be segmented into the environment-aware segmentation model so that the environment-aware segmentation model determines the confidence that the infrared image to be segmented belongs to each environment category, takes the environment category with the highest confidence as the target environment category, and segments the infrared image to be segmented according to the instance segmentation sub-model corresponding to the target environment category to generate an instance mask of the infrared image to be segmented; wherein the environment-aware segmentation model is provided with instance segmentation sub-models corresponding to different environment categories.
7. The instance segmentation device for infrared images of power transmission lines according to claim 6, characterized in that: The instance segmentation module determines the confidence that the infrared image to be segmented belongs to each environment category by: Classifying the infrared image to be segmented according to the environment classification sub-model set in the environment perception segmentation model, and generating a probability value that the infrared image to be segmented belongs to each environment category; According to the gated convolution sub-model built into the environment-aware segmentation model, the probability value of each environment category is calibrated to generate the confidence that the infrared image to be segmented belongs to each environment category.
8. The instance segmentation device for infrared images of power transmission lines according to claim 6, characterized in that: The instance segmentation sub-model is trained as follows: Acquire a plurality of third infrared images of the power transmission line and corresponding semantic labels; wherein the semantic labels are instance masks of the third infrared images; Repeating the training process of the instance segmentation sub-model until a preset third number of executions is reached, the training process of the instance segmentation sub-model comprising: randomly dividing the third infrared image into a plurality of batches of third training samples according to a preset third number; Inputting the batches of third training samples into the current instance segmentation sub-model in sequence, so that when the current instance segmentation sub-model receives each batch of third training samples, it outputs the instance mask corresponding to the third training sample through the current instance segmentation sub-model according to the input third training sample; calculating the third loss function value through the cross entropy loss according to the instance mask and the corresponding semantic label; and updating the current third loss function value according to the third loss function value by using the optimizer; When the preset third execution times is reached, the finally updated instance segmentation sub-model is set as the final instance segmentation sub-model.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the instance segmentation method of the infrared image of the power transmission line as claimed in any one of claims 1 to 5 can be implemented.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the instance segmentation method of the infrared image of the power transmission line described in any one of claims 1 to 5 can be implemented.