Target equipment image segmentation method and system in electric power operation scene

Abstract

The invention discloses a target equipment image segmentation method and system in a power operation scene, and belongs to the technical field of target equipment image segmentation. The method comprises the following steps: acquiring an equipment image of target equipment in a power operation scene; training the equipment image and the loss function to obtain an image semantic segmentation model; and based on the image semantic segmentation model, performing semantic segmentation on the target equipment image of the target equipment in the electric power operation scene to generate a semantic segmentation prediction image. According to the invention, the precision and robustness of the image semantic segmentation model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target device image segmentation, and more specifically, to a target device image segmentation method and system in an electric power operation scenario. Background Art

[0002] In power operation scenarios, accurate image segmentation of key equipment is essential. This not only helps monitor and maintain the health of equipment, but also plays a key role in fault prediction and accident prevention. For example, in intelligent inspection systems, it is necessary to segment power equipment images in real time to quickly identify the different components of the equipment, thereby achieving efficient status monitoring and fault diagnosis.

[0003] Deep learning plays an important role in accurate image segmentation of key equipment in power operation scenarios. Deep learning models can learn the appearance features and structural information of equipment by training a large amount of power equipment image data. This enables the model to automatically and accurately segment key equipment parts in the image, such as the cooling system, insulation layer, wiring, etc. of the transformer. Compared with traditional manually designed features and rules, deep learning models can better adapt to the changes and complexity of different equipment and provide more accurate segmentation results.

[0004] However, deep learning also has some shortcomings in accurate image segmentation of key equipment in power operation scenarios. Deep learning models are usually complex neural network structures with a large number of parameters, resulting in a waste of computing resources. In some cases, image data of power equipment may be limited, especially for specific types of equipment or rare fault conditions. This may lead to insufficient generalization of the model in small sample cases. In addition, the class distribution of different components in power equipment images may be unbalanced, and the number of samples of some parts may be small. This may affect the model's ability to recognize and segment minority classes or rare cases.

[0005] The Chinese patent with publication number CN116862847A discloses "an infrared image interactive segmentation method and system for electric power equipment". By acquiring an image of electric power equipment, the pixel-level label is used as a true mask to form an electric power equipment segmentation data set; using the known true mask, the infrared image and the simulated input information are input into the trained image segmentation model to obtain the predicted mask of the electric power equipment and the segmentation result. Although this method has made some progress in the segmentation of electric power equipment, it is unable to capture and integrate feature information of different sizes, which makes the model's understanding and representation of image details insufficient, resulting in low accuracy and robustness of the model. Summary of the invention

[0006] In view of the above problems, the present invention proposes a target device image segmentation method in a power operation scenario, comprising:

[0007] Obtain equipment images of target equipment in power operation scenarios;

[0008] An image semantic segmentation model is obtained by training with the device image and loss function;

[0009] Based on the image semantic segmentation model, the target device image of the target device in the power operation scenario is semantically segmented to generate a semantic segmentation prediction image.

[0010] Optionally, the image semantic segmentation model is obtained by training the device image and the loss function, including:

[0011] Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio;

[0012] The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set;

[0013] Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map;

[0014] Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image;

[0015] The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

[0016] Optional conversion operations, including:

[0017] Flip operation, rotation operation, crop operation, scale operation and translation operation.

[0018] Optionally, the preset ratio is 9:1.

[0019] Optionally, a multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform a convolution operation on a training set, and through the convolution operation, feature extraction is performed on the training set;

[0020] The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation;

[0021] The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2;

[0022] The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1;

[0023] The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2;

[0024] The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

[0025] Optionally, each position on the feature map of the multi-layer features is connected to other positions through a global view attention model to output a global view attention feature map, including:

[0026] The feature map of the layer features is divided into three branches;

[0027] For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features;

[0028] For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features.

[0029] For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features.

[0030] The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

[0031] Optionally, semantic segmentation is performed on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image, including:

[0032] Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer;

[0033] The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

[0034] On the other hand, the present invention also proposes a target device image segmentation system in a power operation scenario, comprising:

[0035] A data acquisition unit, used to obtain device images of target devices in power operation scenarios;

[0036] A training unit, used for training an image semantic segmentation model using the device image and a loss function;

[0037] The segmentation prediction unit is used to perform semantic segmentation on the target device image of the target device in the power operation scene based on the image semantic segmentation model, and generate a semantic segmentation prediction image.

[0038] Optionally, the training unit obtains an image semantic segmentation model by training the device image and the loss function, including:

[0039] Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio;

[0040] The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set;

[0041] Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map;

[0042] Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image;

[0043] The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

[0044] Optional conversion operations, including:

[0045] Flip operation, rotation operation, crop operation, scale operation and translation operation.

[0046] Optionally, the preset ratio is 9:1.

[0047] Optionally, a multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform a convolution operation on a training set, and through the convolution operation, feature extraction is performed on the training set;

[0048] The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation;

[0049] The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2;

[0050] The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1;

[0051] The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2;

[0052] The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

[0053] Optionally, each position on the feature map of the multi-layer features is connected to other positions through a global view attention model to output a global view attention feature map, including:

[0054] The feature map of the layer features is divided into three branches;

[0055] For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features;

[0056] For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features.

[0057] For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features.

[0058] The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

[0059] Optionally, semantic segmentation is performed on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image, including:

[0060] Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer;

[0061] The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

[0062] In yet another aspect, the present invention further provides a computing device, comprising: one or more processors;

[0063] a processor for executing one or more programs;

[0064] When the one or more programs are executed by the one or more processors, the above-described method is implemented.

[0065] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the method described above is implemented.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention provides a method for segmenting a target device image in an electric power operation scenario, comprising: obtaining a device image of a target device in an electric power operation scenario; training an image semantic segmentation model using the device image and a loss function; and performing semantic segmentation on the target device image of the target device in the electric power operation scenario based on the image semantic segmentation model to generate a semantic segmentation prediction image. The image semantic segmentation model obtained by training the device image and the loss function together improves the accuracy and robustness of the model compared to the existing model trained without using a loss function. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flow chart of the method of the present invention;

[0069] Figure 2 It is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0070] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0071] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0072] Embodiment 1:

[0073] The present invention proposes a target device image segmentation method in a power operation scenario, such as Figure 1 As shown, including:

[0074] Step 1: Obtain a device image of a target device in a power operation scenario;

[0075] Step 2: training an image semantic segmentation model using the device image and loss function;

[0076] Step 3: Based on the image semantic segmentation model, semantic segmentation is performed on the target device image of the target device in the power operation scenario to generate a semantic segmentation prediction image.

[0077] Among them, the generated semantic segmentation prediction image is pixel-level.

[0078] The image semantic segmentation model is obtained by training the device image and the loss function, including:

[0079] Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio;

[0080] The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set;

[0081] Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map;

[0082] Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image;

[0083] The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

[0084] The conversion operation includes:

[0085] Flip operation, rotation operation, crop operation, scale operation and translation operation.

[0086] Among them, the preset ratio is 9:1.

[0087] The multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform convolution operations on the training set, and through the convolution operations, feature extraction is performed on the training set;

[0088] The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation;

[0089] The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2;

[0090] The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1;

[0091] The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2;

[0092] The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

[0093] Among them, through the global field of view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global field of view attention feature map, including:

[0094] The feature map of the layer features is divided into three branches;

[0095] For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features;

[0096] For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features.

[0097] For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features.

[0098] The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

[0099] The semantic segmentation of the global field of view attention feature map is performed using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image, including:

[0100] Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer;

[0101] The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

[0102] The present invention is further described below with reference to specific cases:

[0103] The specific steps include:

[0104] S1. Design an image semantic segmentation model suitable for parsing images of target equipment in power operations;

[0105] S2. Design a loss function used in the process of training the semantic segmentation model of the image of the target equipment of the power operation, and train the image semantic segmentation model designed in step S1;

[0106] The loss function is derived from the fast fusion extraction model loss function L combined composition;

[0107] The fast fusion extraction model loss function L combined The details are as follows:

[0108] L combined =αL bce +βL dice

[0109] Among them, L bce It is used to measure the difference between the probability distribution predicted by the model and the actual label; L dice Used to optimize the model to improve segmentation accuracy; α and β are hyperparameters used to control L bce and L dice The weight of the loss.

[0110] The L bce The details are as follows:

[0111]

[0112] Where N represents the number of pixels; i represents the i-th pixel; p i represents the probability of the prediction of the i-th pixel; g i Represents the true label of the i-th pixel.

[0113] The L dice The details are as follows:

[0114]

[0115] Where N represents the number of pixels; i represents the i-th pixel; p i represents the probability of the prediction of the i-th pixel; g iRepresents the true label of the i-th pixel; smooth is a small constant used to avoid the denominator being zero.

[0116] S3. Use the target device image segmentation model in the power operation scenario trained in step S2 to parse the power operation target device image and generate a pixel-level predicted image.

[0117] Furthermore, the step S1 designs an image semantic segmentation model suitable for parsing the image of the target equipment in the power operation, which includes the following sub-steps:

[0118] S1-1. Collect the target equipment image dataset from the power operation scene through the data preprocessing model. Then, through the data preprocessing model, perform transformation operations such as flipping, rotating, cropping, scaling, and translation on the image dataset, we can expand the size of the original dataset. Finally, divide the dataset into a training set and a test set in a ratio of 9:1;

[0119] S1-2. The training set and test set in step S1-1 are sent to the multi-scale fast fusion extraction model for feature extraction, and multi-layer features of the data in the training set and test set are extracted;

[0120] S1-3. Input the extracted multi-layer features into the global field of view attention model, and connect each position on the feature map to all other positions through the adaptively learned attention mask. In this way, information can flow globally, providing a more comprehensive contextual understanding, thereby improving the accuracy of semantic segmentation;

[0121] S1-4. Input the features output by the global field of view attention model into the semantic segmentation prediction model to obtain the final prediction result.

[0122] Furthermore, the fast fusion extraction model includes 2 convolutional layers and 3 residual structures, including the following sub-steps:

[0123] The image output by the data preprocessing model is input into two convolutional layers for convolution operation. The convolution kernel size of the convolutional layer is 3×3 and the stride of the convolution operation is 2.

[0124] Enter the first residual structure, which includes two convolution kernels. The size of the convolution kernels is 3×3. The stride of the first convolution operation is 2, and the stride of the second convolution operation is 1.

[0125] Enter the second residual structure, which includes three convolution kernels. The size of the convolution kernels is 3×3. The stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2.

[0126] Entering the third residual structure, the third residual structure includes two convolution kernels, the size of each convolution kernel is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

[0127] Furthermore, the global field of view attention model includes the following sub-steps:

[0128] S1-3-1. The features output from the fast fusion extraction model are divided into three branches. The first branch extracts features through two consecutive 3×3 convolutional layers, followed by an average pooling operation to reduce the feature dimension while retaining key information.

[0129] S1-3-2. The second branch is preliminarily processed by a 3×3 convolution layer, then further refined by a 1×1 convolution layer, and finally the most significant feature response is highlighted by a maximum pooling operation;

[0130] S1-3-3. The third branch performs feature compression through a 1×1 convolution layer, followed by a 3×3 convolution layer for feature extraction. After that, average pooling and maximum pooling operations are performed to obtain feature representations of different scales, and finally the two pooling results are spliced ​​to integrate different feature information;

[0131] S1-3-4. Finally, the result outputted by step S1-3-1 and the result outputted by step S1-3-2 are concatenated and inputted into the sigmoid function for normalization. Then, the normalized result is added to the result outputted by step S1-3-3 to obtain the output feature map.

[0132] Furthermore, the semantic segmentation prediction model includes the following sub-steps:

[0133] S1-4-1. Input the output of step S1-3-4 into the 3×3 convolution layer, and change the output size through the 3×3 convolution layer;

[0134] S1-4-2. Input the result outputted by step S1-4-1 into a 1×1 convolution to directly output the final prediction result.

[0135] Furthermore, S3 uses the semantic segmentation model of the power operation target device image trained in step S2 to parse the power operation target device image and generate a pixel-level predicted image, as follows:

[0136] By loading the pre-trained semantic segmentation model of the power operation target equipment image, the power operation target equipment image to be parsed is preprocessed and model inference is performed to generate a pixel-level predicted image.

[0137] Compared with the prior art, the present invention has the following beneficial effects:

[0138] (1) By designing a fast fusion extraction model, which uses convolution operations with multiple strides, the model can effectively capture and integrate feature information of different sizes, thereby improving the model's understanding and representation of image details.

[0139] (2) By designing a global-view attention model, we can combine features of different scales and strengthen the response to important features through the attention mechanism, thereby improving the accuracy and robustness of semantic segmentation while maintaining real-time performance.

[0140] (3) By designing a fast fusion extraction model loss function, the fast fusion extraction model loss function includes Lb ce Loss Function and L dice Loss function, L bce The loss function can help the model better learn and locate the boundaries of objects. For pixels near the boundaries, L bce The loss function can guide the model to accurately predict their categories, thereby improving the boundary accuracy of the segmentation results and improving the accuracy and robustness of the model; L dice The loss function facilitates model optimization and is robust to small objects and class imbalance problems.

[0141] Embodiment 2:

[0142] The present invention also proposes a target device image segmentation system 200 in a power operation scenario, such as Figure 2 As shown, including:

[0143] The data acquisition unit 201 is used to obtain the device image of the target device in the power operation scene;

[0144] A training unit 202, configured to obtain an image semantic segmentation model by training the device image and the loss function;

[0145] The segmentation prediction unit 203 is used to perform semantic segmentation on the target device image of the target device in the power operation scene based on the image semantic segmentation model, and generate a semantic segmentation prediction image.

[0146] The training unit 202 uses the device image and the loss function to train an image semantic segmentation model, including:

[0147] Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio;

[0148] The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set;

[0149] Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map;

[0150] Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image;

[0151] The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

[0152] The conversion operation includes:

[0153] Flip operation, rotation operation, crop operation, scale operation and translation operation.

[0154] Among them, the preset ratio is 9:1.

[0155] The multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform convolution operations on the training set, and through the convolution operations, feature extraction is performed on the training set;

[0156] The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation;

[0157] The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2;

[0158] The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1;

[0159] The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2;

[0160] The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

[0161] Among them, through the global field of view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global field of view attention feature map, including:

[0162] The feature map of the layer features is divided into three branches;

[0163] For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features;

[0164] For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features.

[0165] For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features.

[0166] The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

[0167] The semantic segmentation of the global field of view attention feature map is performed using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image, including:

[0168] Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer;

[0169] The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

[0170] The present invention improves the accuracy and robustness of the image semantic segmentation model.

[0171] Embodiment 3:

[0172] Based on the same inventive concept, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding functions, so as to implement the steps of the method in the above embodiment.

[0173] Embodiment 4:

[0174] Based on the same inventive concept, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both a built-in storage medium in a computer device and an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiment.

[0175] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0180] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A target device image segmentation method in a power operation scenario, characterized in that: include: Obtain equipment images of target equipment in power operation scenarios; An image semantic segmentation model is obtained by training with the device image and loss function; Based on the image semantic segmentation model, the target device image of the target device in the power operation scenario is semantically segmented to generate a semantic segmentation prediction image.

2. The target device image segmentation method according to claim 1, characterized in that: The image semantic segmentation model obtained by training the device image and the loss function includes: Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio; The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set; Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map; Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image; The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

3. The target device image segmentation method according to claim 2, characterized in that: The conversion operation comprises: Flip operation, rotation operation, crop operation, scale operation and translation operation.

4. The target device image segmentation method according to claim 2, characterized in that: The preset ratio is 9:

1.

5. The target device image segmentation method according to claim 2, characterized in that: The multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform convolution operations on the training set, and through the convolution operations, feature extraction is performed on the training set; The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation; The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2; The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1; The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2; The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

6. The target device image segmentation method according to claim 2, characterized in that: The global view attention model is used to connect each position on the feature map of the multi-layer feature with other positions to output the global view attention feature map, including: The feature map of the layer features is divided into three branches; For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features; For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features. For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features. The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

7. The target device image segmentation method according to claim 2, characterized in that: The semantic segmentation of the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image includes: Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer; The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

8. A target equipment image segmentation system in a power operation scenario, characterized in that: include: A data acquisition unit, used to obtain device images of target devices in power operation scenarios; A training unit, used for training an image semantic segmentation model using the device image and a loss function; The segmentation prediction unit is used to perform semantic segmentation on the target device image of the target device in the power operation scene based on the image semantic segmentation model, and generate a semantic segmentation prediction image.

9. The target device image segmentation system according to claim 8, characterized in that: The training unit obtains an image semantic segmentation model by training the device image and the loss function, including: Performing a conversion operation on the device image to expand the data scale of the device image, generating a data set, and dividing the data set into a training set and a test set according to a preset ratio; The training set is sent to a multi-scale fast fusion extraction model, and based on a pre-established loss function, feature extraction is performed on the training set to extract multi-layer features of the training set; Through the global view attention model, each position on the feature map of the multi-layer feature is connected to other positions to output the global view attention feature map; Performing semantic segmentation on the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image; The data processing model, the multi-scale fast fusion extraction model, the global field of view attention model and the semantic segmentation prediction model are fused, the device image is used as the input data of the fused model, the semantic segmentation prediction result is used as the output data of the fused model, the fused model is trained to obtain the image semantic segmentation model, and the accuracy of the image semantic segmentation model is verified using the test set.

10. The target device image segmentation system according to claim 9, characterized in that: The conversion operation comprises: Flip operation, rotation operation, crop operation, scale operation and translation operation.

11. The target device image segmentation system according to claim 9, characterized in that: The preset ratio is 9:

1.

12. The target device image segmentation system according to claim 9, characterized in that: The multi-scale fast fusion model includes: 2 convolutional layers and 3 residual structures, wherein the convolutional layers and the residual structures are used to perform convolution operations on the training set, and through the convolution operations, feature extraction is performed on the training set; The convolution operation includes: inputting the training set into two convolution layers for convolution operation, and inputting the output of the two convolution layers into three residual structures for convolution operation; The convolution kernel size of the convolution layer is 3×3, and the stride of the convolution operation is 2; The first residual structure includes: two convolution kernels, the size of the convolution kernel is 3×3, the stride of the first convolution operation is 2, and the stride of the second convolution operation is 1; The second residual structure includes: three convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second and third convolution operations is 2; The third residual structure includes: two convolution kernels, the convolution kernel size is 3×3, the stride of the first convolution operation is 1, and the stride of the second convolution operation is 2.

13. The target device image segmentation system according to claim 9, characterized in that: The global view attention model is used to connect each position on the feature map of the multi-layer feature with other positions to output the global view attention feature map, including: The feature map of the layer features is divided into three branches; For the first branch, feature extraction is performed through two consecutive 3×3 convolutional layers of the global field of view attention model, and the extracted features are average pooled to reduce the feature dimension and obtain key information features; For the second branch, a 3×3 convolutional layer of the global field of view attention model is used to perform preliminary feature extraction. For the initially extracted features, a 1×1 convolutional layer is used to perform refined feature extraction. The extracted refined features are subjected to the maximum pooling operation to obtain the most significant response features. For the third branch, a 1×1 convolutional layer of the global field of view attention model is used to compress features. For the compressed features, a 3×3 convolutional layer is used to extract features. The extracted features are subjected to average pooling and maximum pooling operations respectively to obtain feature representations of two types of different scales. The feature representations of the two types of different scales are spliced ​​to obtain spliced ​​features. The key information features are concatenated with the most significant response features, and the concatenated features are normalized to obtain normalized concatenated features. The normalized concatenated features are added to the concatenated features so that each position on the feature map of the multi-layer features is connected to other positions to output a global field of view attention feature map.

14. The target device image segmentation system according to claim 9, characterized in that: The semantic segmentation of the global field of view attention feature map using a semantic segmentation prediction model to generate a semantic segmentation prediction result of the device image includes: Inputting the global view attention feature map into a 3×3 convolutional layer of a semantic segmentation prediction model, and adjusting the size of the global view attention feature map in the 3×3 convolutional layer; The resized global view attention feature map is segmented through the 1×1 convolutional layer of the semantic segmentation prediction model to output the semantic segmentation prediction result.

15. A computer device, characterized in that: include: one or more processors; a processor for executing one or more programs; When the one or more programs are executed by the one or more processors, the method according to any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images