Wire image segmentation method and device, equipment and medium

Abstract

The invention provides a wire image segmentation method and device, equipment and a medium. A training wire image is acquired; inputting the training wire image into the original image segmentation model for training to obtain a target image segmentation model; inputting the to-be-segmented image into the target image segmentation model for image segmentation to obtain an image segmentation result; and the image segmentation performance is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image segmentation, and particularly to a method, apparatus, device and medium for wire image segmentation. Background Art

[0002] Currently, in the field of image segmentation of high-voltage wires based on neural networks, the publicly available high-voltage wire datasets are few, and most of the publicly available datasets contain wire towers and insulators in addition to high-voltage wires. Moreover, most of the existing deep neural networks still face problems such as low accuracy in wire segmentation under complex backgrounds. The main reasons are as follows: (1) The high-voltage wires account for a small proportion in the image, and the pixel width of a single wire at a relatively long distance is very small, making it difficult to segment; (2) After multiple downsamplings, some feature information will disappear, and it is easy to ignore the position, texture, edges, etc. of the underlying feature maps, which are crucial feature information for high-voltage wire segmentation; (3) Small targets are affected by various noises during the detection process, and a large number of features generated after the model downsampling are not all effective for small target detection. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of protection of the claims.

[0004] An object of the present application is to solve at least to some extent one of the technical problems existing in the related art. Embodiments of the present application provide a method, apparatus, device and medium for wire image segmentation; and can improve image segmentation performance.

[0005] In an embodiment of the first aspect of the present application, a method for wire image segmentation includes:

[0006] Obtain a training wire image;

[0007] Input the training wire image into an original image segmentation model for training to obtain a target image segmentation model;

[0008] Input an image to be segmented into the target image segmentation model for image segmentation to obtain an image segmentation result.

[0009] According to the embodiment of the first aspect of the present application, the step of inputting the training wire image into an original image segmentation model for training to obtain a target image segmentation model includes:

[0010] Perform preliminary image segmentation on the training wire image to obtain a preliminary semantic segmentation result;

[0011] Perform an attention mechanism on the training wire image to obtain an attention image;

[0012] Segment and reorganize according to the preliminary semantic segmentation result to obtain a feature matrix;

[0013] Element - wise multiply the feature matrix and the attention image to obtain a masked image;

[0014] Add the masked image and the preliminary semantic segmentation result to obtain a target segmentation image;

[0015] Adjust the parameters of the original image segmentation model according to the target segmentation image to obtain a target image segmentation model.

[0016] According to an embodiment of the first aspect of the present application, the performing preliminary image segmentation on the training wire image to obtain a preliminary semantic segmentation result includes:

[0017] Perform a convolution operation, an activation operation, and a downsampling operation on the training wire image in sequence to obtain a first feature map;

[0018] Perform a convolution operation, an activation operation, and a downsampling operation on the first feature map in sequence to obtain a second feature map;

[0019] Perform a convolution operation, an activation operation, and a downsampling operation on the second feature map in sequence to obtain a third feature map;

[0020] Perform a convolution operation, an activation operation, and a downsampling operation on the third feature map in sequence to obtain a fourth feature map;

[0021] Perform a convolution operation, an activation operation, and a downsampling operation on the fourth feature map in sequence to obtain a fifth feature map;

[0022] Perform a convolution operation and an activation operation on the fifth feature map in sequence to obtain a bottleneck feature map;

[0023] Perform an upsampling operation on the bottleneck feature map in sequence to obtain a first upsampled feature map, splice the first upsampled feature map and the fourth feature map, and then perform a convolution operation and an activation operation in sequence to obtain a sixth feature map;

[0024] Perform an upsampling operation on the sixth feature map in sequence to obtain a second upsampled feature map, splice the second upsampled feature map and the third feature map, and then perform a convolution operation and an activation operation in sequence to obtain a seventh feature map;

[0025] Perform an upsampling operation on the seventh feature map in sequence to obtain a third upsampled feature map, splice the third upsampled feature map and the second feature map, and then perform a convolution operation and an activation operation in sequence to obtain an eighth feature map;

[0026] Perform an upsampling operation on the eighth feature map in sequence to obtain a fourth upsampled feature map, splice the fourth upsampled feature map with the first feature map, and then perform a convolution operation and an activation operation in sequence to obtain a ninth feature map;

[0027] Perform a convolution operation and an activation operation on the ninth feature map to obtain a preliminary semantic segmentation result.

[0028] According to an embodiment of the first aspect of the present application, the performing an attention mechanism on the training wire image to obtain an attention image includes:

[0029] Perform a convolution operation on the training wire image to obtain a convolution feature map, and divide the convolution feature map into multiple first image blocks;

[0030] Perform a convolution operation on the second feature map to obtain a channel feature map with two channels, divide the channel feature map into multiple second image blocks, and splice the second image blocks into a second feature matrix;

[0031] Perform a dot product operation on the first image block and the second feature matrix to obtain an attention image.

[0032] According to an embodiment of the first aspect of the present application, the segmenting and reorganizing according to the preliminary semantic segmentation result to obtain a feature matrix includes:

[0033] Perform an average pooling operation on the preliminary semantic segmentation result to obtain an average pooling feature;

[0034] Divide the average pooling feature into multiple third image blocks of the same size;

[0035] Reorganize multiple third image blocks into a feature matrix.

[0036] According to an embodiment of the first aspect of the present application, the adding the mask image and the preliminary semantic segmentation result to obtain a target segmentation image includes:

[0037] Perform an unfolding operation on the mask image to obtain a second mask image;

[0038] Add the second mask image and the preliminary semantic segmentation result to obtain a target segmentation image.

[0039] According to an embodiment of the first aspect of the present application, the adjusting the parameters of the original image segmentation model according to the target segmentation image to obtain a target image segmentation model includes:

[0040] Adjust the parameters of the original image segmentation model through a loss function to obtain a target image segmentation model;

[0041] Among them, the loss function is expressed as: In the formula, is the total value of the loss function, is the value of the standard segmentation loss function, is the value of the mean square error loss function, CrossEntropy is the cross-entropy operation, s pred is the predicted value of the target segmentation image, s gt is the true value corresponding to the target segmentation image, MSE is the mean square error operation, A ith is the attention image, A gt is the training wire image.

[0042] In an embodiment of the second aspect of the present application, a wire image segmentation device includes:

[0043] An input unit for acquiring a training wire image;

[0044] A training unit for inputting the training wire image into an original image segmentation model for training to obtain a target image segmentation model;

[0045] A segmentation unit for inputting an image to be segmented into the target image segmentation model for image segmentation to obtain an image segmentation result.

[0046] In an embodiment of the third aspect of the present application, an electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the wire image segmentation method as described in the embodiment of the first aspect of the present application.

[0047] In an embodiment of the fourth aspect of the present application, a computer storage medium stores computer-executable instructions for executing the wire image segmentation method as described in the embodiment of the first aspect of the present application.

[0048] The above solution has at least the following beneficial effects: By quantifying the relationship between different feature maps, especially the similarity between small objects and large objects, it effectively compensates for the information loss caused by convolution and pooling operations; this makes the model more sensitive when capturing the details of high-voltage wires and improves the detection ability of the slender structure of the wires. By calculating the cross-level relationship between the feature map and the original image block, it can effectively fuse the low-level detail features and high-level semantic features, making the edges and details of the wires clearer in a complex background. Description of the Drawings

[0049] The accompanying drawings are used to provide a further understanding of the technical solution of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application, and do not constitute a limitation to the technical solution of the present application.

[0050] Figure 1 It is a step diagram of the wire image segmentation method provided by the embodiment of the present application;

[0051] Figure 2 It is a sub-step diagram of step S200;

[0052] Figure 3 It is a sub-step diagram of step S220;

[0053] Figure 4 It is a sub-step diagram of step S230;

[0054] Figure 5 It is a sub-step diagram of step S250;

[0055] Figure 6 It is a structural diagram of UNet;

[0056] Figure 7 It is a schematic diagram of step S220;

[0057] Figure 8 It is a schematic diagram of steps S230 to S250;

[0058] Figure 9 It is a structural diagram of the wire image segmentation device provided by the embodiment of the present application. Detailed implementation manners

[0059] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division in the device or a different order in the flowchart. Terms such as "first" and "second" in the specification, claims or the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0061] The following further elaborates on the embodiments of the present application with reference to the accompanying drawings.

[0062] The embodiment of the present application provides a wire image segmentation method.

[0063] Refer to Figure 1, A wire image segmentation method, including but not limited to the following steps:

[0064] Step S100, obtaining training wire images;

[0065] Step S200, inputting the training wire images into an original image segmentation model for training to obtain a target image segmentation model;

[0066] Step S300, inputting the image to be segmented into the target image segmentation model for image segmentation to obtain an image segmentation result.

[0067] For step S100, obtaining training wire images. Taking real-time wire images through a camera and performing label annotation on the real-time wire images; alternatively, obtaining wire images from the image database of the network.

[0068] Performing image preprocessing on the wire images; the image preprocessing includes processing such as filtering, rotation, and cropping.

[0069] Dividing the preprocessed wire images into a training set and a validation set. The wire images in the training set are the training wire images.

[0070] The training wire images contain various wires, including high-voltage wires. Performing label annotation on the high-voltage wires in the training wire images.

[0071] Refer to Figure 2 , For step S200, inputting the training wire images into an original image segmentation model for training to obtain a target image segmentation model, including but not limited to the following steps:

[0072] Step S210, performing preliminary image segmentation on the training wire images to obtain a preliminary semantic segmentation result;

[0073] Step S220, performing an attention mechanism on the training wire images to obtain an attention image;

[0074] Step S230, performing segmentation and recombination according to the preliminary semantic segmentation result to obtain a feature matrix;

[0075] Step S240, multiplying the feature matrix and the attention image element by element to obtain a mask image;

[0076] Step S250, adding the mask image and the preliminary semantic segmentation result to obtain a target segmentation image;

[0077] Step S260, adjusting the parameters of the original image segmentation model according to the target segmentation image to obtain a target image segmentation model.

[0078] Performing preliminary image segmentation on the training wire images to obtain a preliminary semantic segmentation result, including the following steps:

[0079] Perform convolution operations, activation operations, and downsampling operations on the training wire image in sequence to obtain the first feature map;

[0080] Perform convolution operations, activation operations, and downsampling operations on the first feature map in sequence to obtain the second feature map;

[0081] Perform convolution operations, activation operations, and downsampling operations on the second feature map in sequence to obtain the third feature map;

[0082] Perform convolution operations, activation operations, and downsampling operations on the third feature map in sequence to obtain the fourth feature map;

[0083] Perform convolution operations, activation operations, and downsampling operations on the fourth feature map in sequence to obtain the fifth feature map;

[0084] Perform convolution operations and activation operations on the fifth feature map in sequence to obtain the bottleneck feature map;

[0085] Perform upsampling operations on the bottleneck feature map in sequence to obtain the first upsampled feature map, splice the first upsampled feature map with the fourth feature map, and then perform convolution operations and activation operations in sequence to obtain the sixth feature map;

[0086] Perform upsampling operations on the sixth feature map in sequence to obtain the second upsampled feature map, splice the second upsampled feature map with the third feature map, and then perform convolution operations and activation operations in sequence to obtain the seventh feature map;

[0087] Perform upsampling operations on the seventh feature map in sequence to obtain the third upsampled feature map, splice the third upsampled feature map with the second feature map, and then perform convolution operations and activation operations in sequence to obtain the eighth feature map;

[0088] Perform upsampling operations on the eighth feature map in sequence to obtain the fourth upsampled feature map, splice the fourth upsampled feature map with the first feature map, and then perform convolution operations and activation operations in sequence to obtain the ninth feature map;

[0089] Perform convolution operations and activation operations on the ninth feature map to obtain the preliminary semantic segmentation result.

[0090] Specifically, perform preliminary image segmentation on the training wire image through UNet.

[0091] Refer to Figure 6 , pass the training wire image with a size of 512*512*3 through the first convolutional module of the contraction path. The first convolutional module includes two consecutive 3x3 convolution operations. After each convolution operation, connect the ReLU activation function, and then perform downsampling through a 2x2 max pooling layer. Obtain the first feature map with a size of 512*512*32.

[0092] Pass the first feature map with dimensions 512*212*32 through the second convolutional module of the contracting path. The second convolutional module consists of two consecutive 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied, and then downsampling is performed through a 2x2 max pooling layer. A second feature map with dimensions 256*256*64 is obtained.

[0093] Pass the second feature map with dimensions 256*256*64 through the third convolutional module of the contracting path. The third convolutional module consists of two consecutive 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied, and then downsampling is performed through a 2x2 max pooling layer. A third feature map with dimensions 128*128*128 is obtained.

[0094] Pass the third feature map with dimensions 128*128*128 through the fourth convolutional module of the contracting path. The fourth convolutional module consists of two consecutive 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied, and then downsampling is performed through a 2x2 max pooling layer. A fourth feature map with dimensions 64*64*256 is obtained.

[0095] Pass the fourth feature map with dimensions 64*64*256 through the fifth convolutional module of the contracting path. The fifth convolutional module consists of two consecutive 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied, and then downsampling is performed through a 2x2 max pooling layer. A fifth feature map with dimensions 32*32*512 is obtained.

[0096] Pass the fifth feature map with dimensions 32*32*512 through two 3x3 convolutional operations, and then through a ReLU activation function to obtain a bottleneck feature map with dimensions 32*32*1024.

[0097] Pass the bottleneck feature map with dimensions 32*32*1024 through the expanding path. First, perform a 2x2 upsampling operation to obtain the first upsampled feature map. Concatenate the first upsampled feature map with the fourth feature map, and then perform two 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied to obtain a sixth feature map with dimensions 64*64*512.

[0098] Perform a 2x2 upsampling operation on the sixth feature map with dimensions 64*64*512 to obtain the second upsampled feature map. Concatenate the second upsampled feature map with the third feature map, and then perform two 3x3 convolutional operations. After each convolutional operation, a ReLU activation function is applied to obtain a seventh feature map with dimensions 128*128*256.

[0099] The seventh feature map with a size of 128*128*256 is first subjected to a 2x2 upsampling operation to obtain the third upsampled feature map. The third upsampled feature map is concatenated with the second feature map, and then two 3x3 convolution operations are performed, with a ReLU activation function connected after each convolution operation, to obtain the eighth feature map with a size of 256*256*128.

[0100] The eighth feature map with a size of 256*256*128 is first subjected to a 2x2 upsampling operation to obtain the fourth upsampled feature map. The fourth upsampled feature map is concatenated with the first feature map, and then two 3x3 convolution operations are performed, with a ReLU activation function connected after each convolution operation, to obtain the ninth feature map with a size of 512*512*64.

[0101] A 1x1 convolution operation is performed on the ninth feature map with a size of 512*512*64 to generate a feature image, and the number of channels of the feature image is equal to the number of classification categories. The feature image is passed through a soft-max activation function to obtain a preliminary semantic segmentation result.

[0102] Refer to Figure 3 , for step S220, performing an attention mechanism on the training wire image to obtain an attention image, including but not limited to the following steps:

[0103] Step S221, performing a convolution operation on the training wire image to obtain a convolution feature map, and dividing the convolution feature map into multiple first image blocks;

[0104] Step S222, performing a convolution operation on the second feature map to obtain a channel feature map with two channels, dividing the channel feature map into multiple second image blocks, and concatenating the second image blocks into a second feature matrix;

[0105] Step S223, performing a dot product operation on the first image block and the second feature matrix to obtain an attention image.

[0106] Specifically, performing a 7x7 convolution operation and a 3x3 convolution operation on the training wire image to obtain a convolution feature map, and dividing the convolution feature map into multiple first image blocks of 32x32. The first image blocks are concatenated into a first feature matrix.

[0107] Performing a 1x1 convolution operation on the second feature map to obtain a channel feature map with two channels, dividing the channel feature map into multiple second image blocks of 32x32. The second image blocks are concatenated into a second feature matrix.

[0108] Performing a dot product operation on the first image block and the second feature matrix to obtain an attention image.

[0109] Specifically, refer to Figure 7, perform 7x7 convolution operation and 3x3 convolution operation on the training wire image with a size of 512*512*3 to obtain a convolution feature map, and then perform segmentation to obtain a first feature matrix with a size of 256*1024*1 and composed of multiple first image blocks;

[0110] Perform 1x1 convolution operation on the second feature map to obtain a channel feature map with two channels, and divide the channel feature map into multiple 32x32 second image blocks. Stitch the second image blocks into a second feature matrix with a size of 64*1024*2.

[0111] Perform dot product operation on the first feature matrix and the second feature matrix to obtain an attention image.

[0112] Refer to Figure 4 , for step S230, perform segmentation and recombination according to the preliminary semantic segmentation result to obtain a feature matrix, including but not limited to the following steps:

[0113] Step S231, perform average pooling operation on the preliminary semantic segmentation result to obtain an average pooling feature;

[0114] Step S232, divide the average pooling feature into multiple third image blocks with the same size;

[0115] Step S233, recombine multiple third image blocks into a feature matrix.

[0116] For step S240, perform element-wise multiplication on the feature matrix and the attention image to obtain a mask image.

[0117] Refer to Figure 5 , for step S250, add the mask image and the preliminary semantic segmentation result to obtain a target segmentation image, including but not limited to the following steps:

[0118] Step S251, perform unfolding operation on the mask image to obtain a second mask image;

[0119] Step S252, add the second mask image and the preliminary semantic segmentation result to obtain a target segmentation image.

[0120] Specifically, refer to Figure 8, perform average pooling operation on the preliminary semantic segmentation result with a size of 512*512*2 to obtain average pooling features, split the average pooling features into multiple third image patches of the same size, and recombine the multiple third image patches into a feature matrix with a size of 64*1024*2. Multiply the feature matrix with a size of 64*1024*2 element-wise with the attention image with a size of 256*64*2 to obtain a mask image. Perform an unfolding operation on the mask image to obtain a second mask image with a size of 512*512*2. Add the second mask image with a size of 512*512*2 to the preliminary semantic segmentation result with a size of 512*512*2 to obtain a target segmentation image with a size of 512*512*2.

[0121] By quantifying the relationships between different feature maps, especially the similarities between small objects and large objects, the information loss caused by convolution and pooling operations is effectively compensated; this makes the model more sensitive when capturing the details of high-voltage wires and improves the detection ability of the slender structure of the wires. By calculating the cross-level relationships between the feature maps and the original image patches, the low-level detail features and high-level semantic features can be effectively fused, making the edges and details of the wires clearer in complex backgrounds.

[0122] For step S260, adjusting the parameters of the original image segmentation model according to the target segmentation image to obtain a target image segmentation model includes, but is not limited to, the following steps:

[0123] Adjust the parameters of the original image segmentation model through a loss function to obtain a target image segmentation model;

[0124] Among them, the loss function is expressed as: In the formula, L is the total value of the loss function, is the value of the standard segmentation loss function, is the value of the mean square error loss function, CrossEntropy is the cross-entropy operation, s pred is the predicted value of the target segmentation image, s gt is the true value corresponding to the target segmentation image, MSE is the mean square error operation, A ith is the attention image, A gt is the training wire image.

[0125] Minimize the difference between the prediction and the standard segmentation mask through the standard segmentation loss function. Minimize the difference between the learned cross-feature map attention mechanism and the standard cross-feature map attention mechanism through the mean square error loss function for the cross-feature map attention mechanism. Among them, the standard cross-feature map attention mechanism is calculated based on the original image and the corresponding standard segmentation mask, and provides supervision for the cross-feature map attention mechanism attention map to optimize the quality of the attention map.

[0126] After training the target image segmentation model, the target image segmentation model is verified using the validation set.

[0127] For step S300, the image to be segmented is input into the target image segmentation model for image segmentation to obtain the image segmentation result.

[0128] The image to be segmented is an image without label annotation. The image to be segmented is input into the target image segmentation model. The target image segmentation model performs image segmentation on the image to be segmented and outputs the image segmentation result. The image segmentation result includes the prediction box for the high-voltage wire, the coordinates of the prediction box, and the classification type of the high-voltage wire corresponding to the prediction box.

[0129] An embodiment of the present application provides a wire image segmentation device.

[0130] Referring to Figure 9 , the wire image segmentation device includes: an input unit 10, a training unit 20, and a segmentation unit 30.

[0131] Among them, the input unit 10 is used to obtain the training wire image; the training unit 20 is used to input the training wire image into the original image segmentation model for training to obtain the target image segmentation model; the segmentation unit 30 is used to input the image to be segmented into the target image segmentation model for image segmentation to obtain the image segmentation result.

[0132] It can be understood that the wire image segmentation device provided in this embodiment applies the above-mentioned wire image segmentation method. Each unit of the wire image segmentation device corresponds to each step of the above-mentioned wire image segmentation method one by one. The wire image segmentation device and the wire image segmentation method adopt the same technical means, solve the same technical problems, and have the same technical effects.

[0133] An embodiment of the present application provides an electronic device. The electronic device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned wire image segmentation method is implemented.

[0134] The electronic device can be any intelligent terminal including a computer, etc.

[0135] Generally speaking, for the hardware structure of the electronic device, the processor can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0136] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory and are called by the processor to execute the methods of the embodiments of this application.

[0137] The input / output interface is used to implement information input and output.

[0138] The communication interface is used to implement the communication interaction between this device and other devices. It can implement communication through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0139] The bus transmits information between various components of the device (such as the processor, the memory, the input / output interface, and the communication interface). The processor, the memory, the input / output interface, and the communication interface are communicatively connected to each other inside the device through the bus.

[0140] Embodiments of this application provide a computer storage medium. The computer storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the wire image segmentation method as described above.

[0141] Those of ordinary skill in the art will understand that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented as hardware, or can be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. In the foregoing description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example", or "certain embodiments / examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0142] Those of ordinary skill in the art will understand that all or some of the steps, systems, and functional modules / units in the devices disclosed above can be implemented as software, firmware, hardware, and their appropriate combinations.

[0143] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The aforementioned storage medium includes: various media that can store programs, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0146] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms. Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of the present application. The scope of the present application is defined by the claims and their equivalents.

[0147] The above has specifically described the preferred embodiments of the present application, but the present application is not limited to the embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present application, and these equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A wire image segmentation method, characterized in that: include: Get training wire images; Inputting the training wire image into the original image segmentation model for training to obtain a target image segmentation model; The image to be segmented is input into the target image segmentation model for image segmentation to obtain an image segmentation result.

2. The wire image segmentation method according to claim 1, characterized in that: The step of inputting the training wire image into the original image segmentation model for training to obtain a target image segmentation model comprises: Performing preliminary image segmentation on the training wire image to obtain preliminary semantic segmentation results; Performing an attention mechanism on the training wire image to obtain an attention image; Segment and reorganize the result of the preliminary semantic segmentation to obtain a feature matrix; Multiplying the feature matrix by the attention image element by element to obtain a mask image; Adding the mask image to the preliminary semantic segmentation result to obtain a target segmentation image; The parameters of the original image segmentation model are adjusted according to the target segmented image to obtain a target image segmentation model.

3. The wire image segmentation method according to claim 2, characterized in that: The performing preliminary image segmentation on the training wire image to obtain a preliminary semantic segmentation result includes: Performing convolution operation, activation operation and downsampling operation on the training wire image in sequence to obtain a first feature map; Performing a convolution operation, an activation operation, and a downsampling operation on the first feature map in sequence to obtain a second feature map; Performing a convolution operation, an activation operation, and a downsampling operation on the second feature map in sequence to obtain a third feature map; Performing a convolution operation, an activation operation, and a downsampling operation on the third feature map in sequence to obtain a fourth feature map; Performing a convolution operation, an activation operation, and a downsampling operation on the fourth feature map in sequence to obtain a fifth feature map; Performing convolution operations and activation operations on the fifth feature map in sequence to obtain a bottleneck feature map; Performing upsampling operations on the bottleneck feature maps in sequence to obtain a first upsampling feature map, concatenating the first upsampling feature map with the fourth feature map, and then performing convolution operations and activation operations in sequence to obtain a sixth feature map; Performing upsampling operations on the sixth feature map in sequence to obtain a second upsampling feature map, concatenating the second upsampling feature map with the third feature map, and then performing convolution operations and activation operations in sequence to obtain a seventh feature map; Performing upsampling operations on the seventh feature map in sequence to obtain a third upsampling feature map, concatenating the third upsampling feature map with the second feature map, and then performing convolution operations and activation operations in sequence to obtain an eighth feature map; Performing upsampling operations on the eighth feature map in sequence to obtain a fourth upsampling feature map, concatenating the fourth upsampling feature map with the first feature map, and then performing convolution operations and activation operations in sequence to obtain a ninth feature map; The ninth feature map is subjected to convolution and activation operations to obtain a preliminary semantic segmentation result.

4. The wire image segmentation method according to claim 3, characterized in that: The step of performing an attention mechanism on the training wire image to obtain an attention image includes: Performing a convolution operation on the training wire image to obtain a convolution feature map, and dividing the convolution feature map into a plurality of first image blocks; Performing a convolution operation on the second feature map to obtain a channel feature map having two channels, dividing the channel feature map into a plurality of second image blocks, and splicing the second image blocks into a second feature matrix; Perform a dot product operation on the first image block and the second feature matrix to obtain an attention image.

5. The wire image segmentation method according to claim 2, characterized in that: The segmentation and reorganization according to the preliminary semantic segmentation result to obtain a feature matrix includes: Performing an average pooling operation on the preliminary semantic segmentation result to obtain an average pooling feature; Dividing the average pooled feature into a plurality of third image blocks of the same size; The plurality of third image blocks are reorganized into a feature matrix.

6. The wire image segmentation method according to claim 2, characterized in that: The step of adding the mask image to the preliminary semantic segmentation result to obtain a target segmentation image includes: Performing an unfolding operation on the mask image to obtain a second mask image; The second mask image is added to the preliminary semantic segmentation result to obtain a target segmentation image.

7. The wire image segmentation method according to claim 2, characterized in that: The step of adjusting the parameters of the original image segmentation model according to the target segmented image to obtain the target image segmentation model includes: Adjusting the parameters of the original image segmentation model through the loss function to obtain a target image segmentation model; Wherein, the loss function is expressed as: In the formula, is the total value of the loss function, is the value of the standard segmentation loss function, is the value of the mean square error loss function, CrossEntropy is the cross entropy operation, s pred is the predicted value of the target segmentation image, s gt is the true value corresponding to the target segmentation image, MSE is the mean square error operation, A ith is the attention image, A gt For training wire images.

8. A wire image segmentation device, characterized in that: include: An input unit, used to obtain training wire images; A training unit, used for inputting the training wire image into the original image segmentation model for training to obtain a target image segmentation model; The segmentation unit is used to input the image to be segmented into the target image segmentation model to perform image segmentation and obtain an image segmentation result.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the wire image segmentation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the wire image segmentation method according to any one of claims 1 to 7.

Images