A single target object segmentation method and detection device based on deep learning

Abstract

The present invention belongs to the technical field of image processing in the detection of the height of the groove of the motor retaining spring, and discloses a single target object segmentation method and detection equipment based on deep learning, which uses an improved U-Net neural network to segment the complete target and filter out the complete target when the same type of incomplete target and complete target exist at the same time. The present invention improves the U-Net neural network, and when the same type of incomplete target and complete target exist at the same time, the network can directly segment the complete target and filter out the incomplete target. This method is of great help when only the complete target is considered, reduces the interference of the incomplete target, and shortens the complete target segmentation steps; and the improved U-Net network improves the segmentation efficiency of the network compared to the original U-Net network.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing in motor retaining spring groove height detection, and in particular relates to a single target object segmentation method based on deep learning. Background Art

[0002] At present, with the continuous development of science and technology, the degree of automation of industrial production lines is constantly improving, and more and more industrial production lines are beginning to use machine vision technology to assist workers in their work, such as target detection, defect detection, object segmentation, etc. For object segmentation, image segmentation and instance segmentation technology are mainly used, that is, different objects are defined as different labels, and then they are segmented. The current deep learning segmentation network mainly considers the image segmentation of multiple targets and multiple types of targets, but ignores the segmentation of complete targets when complete targets and incomplete targets of the same type exist at the same time, that is, single target object segmentation.

[0003] Patent number CN110930419A discloses an image segmentation method, device, electronic device and computer storage medium, which introduces a method for segmenting a single target in a multi-target image, first performing target detection, and then semantically segmenting the target detection area. The invention has complex segmentation steps, requires two deep learning models, has low segmentation efficiency, and does not consider how to segment when the segmented target is incomplete.

[0004] Patent number CN111080613A discloses a method for identifying images of damaged bathtubs in railway freight cars. Although the patent improves the U-Net network and increases the prediction time, it is still unable to handle the segmentation situation where complete and incomplete targets of the same type coexist.

[0005] Through the above analysis, the problems and defects of the prior art are as follows:

[0006] (1) In the detection of the groove height of the motor retaining spring, when the same type of complete and incomplete targets exist at the same time, the current image segmentation network cannot only segment the complete targets, but can only segment all targets. And it cannot filter out the incomplete targets.

[0007] (2) If the steps of first detecting the target and then segmenting the target are used, the overall segmentation efficiency will be slowed down, the model complexity cannot be reduced, and the target segmentation time will be prolonged. Summary of the invention

[0008] In view of the problems existing in the prior art, the present invention provides a single target object segmentation method based on deep learning.

[0009] The first object of the present invention is that when the same type of complete and incomplete targets exist at the same time, the current image segmentation network cannot only segment out the complete target, but can only segment out all the targets. The segmentation method proposed in the present invention can directly segment out the complete target and filter out the incomplete target.

[0010] The second purpose of the present invention is that if the steps of first detecting the target and then segmenting the target are used, the overall segmentation efficiency will be slowed down. The segmentation method proposed in the present invention does not need to perform target detection before segmentation, and the present invention reduces the model complexity, shortens the target segmentation time, and improves the segmentation efficiency.

[0011] The present invention is implemented as follows: a single target object segmentation method based on deep learning, comprising:

[0012] Using the improved U-Net neural network, when incomplete targets and complete targets of the same type coexist, the complete targets are segmented and the incomplete targets are filtered out.

[0013] Furthermore, the improved U-net network includes: multiple decoders with reduced numbers of partial convolution kernels, and the number of convolution kernels of each decoder is reduced to 4 times of the original number; the reduced decoder reduces the features obtained by the model, so that the model does not pay attention to the detailed features of the target, overfits the complete target, only segments the complete target in the ROI area, and ignores the incomplete target.

[0014] Further, when the same type of incomplete target and complete target exist at the same time, the complete target and the incomplete target are the same type of object;

[0015] In the complete target segmentation, the target image has one and only one complete target, and the ROI area is extracted before segmentation; the extracted ROI area only contains one complete target, and there is no restriction on the incomplete targets contained.

[0016] Furthermore, the ROI region extraction method includes roughly determining the size of the region containing the complete target according to the target center, and determining the size of the ROI region according to the target contour, wherein the ROI region includes a plurality of incomplete targets;

[0017] Then the ROI area image is input into the improved U-net network.

[0018] Furthermore, the complete target segmentation method specifically includes:

[0019] (1) Perform image acquisition;

[0020] (2) The collected image is used to select the ROI region according to the location of the target, and the ROI region image is input into the improved U-Net image segmentation network to segment the image mask of a single target;

[0021] (3) The single target image mask is fused with the original image, and the fused image is the single target segmentation image.

[0022] Furthermore, the fusion method of step (3) includes: changing the pixel value of the complete target object position in the mask to 1, and the pixel values ​​of the remaining positions to 0, and then multiplying with the original image, so that only the single target image is retained in the original image, and the remaining positions are all changed in color; the fused image only has one complete target object, and the single target image segmentation is completed.

[0023] Another object of the present invention is to provide an improved U-Net neural network for implementing the single target object segmentation method based on deep learning.

[0024] Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0025] Using the improved U-Net neural network, when incomplete targets and complete targets of the same type coexist, the complete targets are segmented and the incomplete targets are filtered out.

[0026] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0027] Using the improved U-Net neural network, when incomplete targets and complete targets of the same type coexist, the complete targets are segmented and the incomplete targets are filtered out.

[0028] Another object of the present invention is to provide a motor retaining spring groove height detection device that implements the single target object segmentation method based on deep learning.

[0029] Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows: by improving the U-Net neural network, when incomplete targets and complete targets of the same type exist at the same time, the network can directly segment the complete target and filter out the incomplete target. This method is very helpful when only considering the complete target situation, reducing the interference of incomplete targets and shortening the complete target segmentation steps; and the improved U-Net network improves the segmentation efficiency of the network compared to the original U-Net network.

[0030] Compared with the prior art, the advantages of the present invention further include:

[0031] The present invention is to improve the decoder part of the U-net network, so as to achieve that when the same type of incomplete and complete targets exist at the same time, only the complete target can be segmented. The premise of the method is that there is and only one complete target in the image, so it is necessary to extract the ROI area before segmentation, and the ROI area principle is that the ROI area after extraction only contains one complete target, and there is no restriction on the incomplete target contained, and the complete target and the incomplete target are of the same type, i.e., the same object. The original U-net network, when the complete target and the incomplete target exist at the same time, cannot separate the complete target alone, but only separates all the complete and incomplete targets from the background. In this case, when it is necessary to judge certain features in the complete target, it will be impossible to judge, so the present invention is adopted, only the complete target can be segmented out, and the incomplete target is eliminated, so as to achieve the purpose of performing corresponding operations in the complete target. There are multiple methods for extracting the ROI area, including roughly judging the size of the area containing the complete target according to the target center, judging the size of the ROI area according to the target contour, etc., and multiple incomplete targets can be included in the ROI area. Then the ROI area image is input into the improved U-net network. The improvement of the U-net network by the present invention is to reduce the number of convolution kernels of the decoder part of the network, and the number of convolution kernels of each decoder is reduced to 4 times of the original number. Of course, this multiple is not limited. The reason for reducing the decoder is to reduce the features obtained by the model, so that the model does not pay attention to the detailed features of the target and overfits the complete target. Because the model only recognizes the complete target (overfitting), the model will only segment out the complete target in the ROI area, and ignore the incomplete target, so as to achieve the purpose of segmenting a single target. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 It is a flow chart of a single target object segmentation method based on deep learning provided in an embodiment of the present invention.

[0034] Figure 2 It is a schematic diagram of the structure of the U-Net network decoder provided in an embodiment of the present invention.

[0035] Figure 2Middle: 1 is the original U-Net network decoder structure, 2 is the improved U-Net network decoder structure.

[0036] Figure 3 It is a schematic diagram of a single target object provided by an embodiment of the present invention.

[0037] Figure 3 In the figure, 1 is a complete target object, and 2, 3, and 4 are incomplete target objects. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] In view of the problems existing in the prior art, the present invention provides a single target object segmentation method based on deep learning, and the present invention is described in detail below in conjunction with the accompanying drawings.

[0040] The single target object segmentation method based on deep learning provided by the present invention reduces the number of convolution kernels in the U-Net network decoder structure so that the network only grasps the features of the complete target, thereby realizing the segmentation of the single target object. By segmenting the single target object, the steps of target detection and positioning are reduced, and the target segmentation can be directly performed, which simplifies the target segmentation scheme and improves the target segmentation efficiency.

[0041] like Figure 1 As shown, the present invention provides a single target object segmentation method based on deep learning, comprising:

[0042] Using the improved U-Net neural network, when incomplete targets and complete targets of the same type coexist, the complete targets are segmented and the incomplete targets are filtered out.

[0043] The single target object segmentation method based on deep learning provided by the present invention can also be implemented by other steps by ordinary technicians in the industry. Figure 1 It is only a specific embodiment provided by the present invention.

[0044] In the present invention, a solution is provided for the segmentation of a single target object. First, image acquisition is performed, and then the ROI region is selected according to the position of the target in the acquired image, and the ROI region image is input into the improved U-Net image segmentation network to segment the image mask of the single target, and then the single target image mask is fused with the original image, and the fused image is the single target segmentation image.

[0045] In the present invention, image acquisition is first required. The acquired image should ensure that the target object is clearly visible and contains at least one complete target in the image, otherwise single target object segmentation cannot be performed. The principle of ROI area selection is that the image contains only one complete target object, and incomplete target objects are not restricted. The target objects here refer to objects of the same type. A variety of methods can be used to select the ROI area, such as adjusting the camera shooting position so that a complete target is included in the camera field of view.

[0046] For the U-Net image segmentation network, the present invention makes the following improvements: the encoder part is not modified so that the segmentation network can fully extract the features of the target object, and then in the decoder part, the data of the convolution kernel is reduced, such as Figure 2 As shown in Figure 2, the number of convolution kernels is reduced by 4 times compared to the original U-Net network in each decoder module. After the improvement, the U-Net segmentation model can be used in Figure 3 In the case shown, the mask of the complete target object is directly segmented and the incomplete target object is filtered out.

[0047] In the image segmentation stage, the single target segmentation model needs to be trained in advance. The improved U-Net model reduces the number of convolution kernels, reduces model parameters during training, and improves the model segmentation efficiency. Then the ROI area image that needs to be segmented can be input into the segmentation network model. The model will output the mask of the single target object. Next, the output mask can be fused with the original image. There are also many fusion methods. Here is a method that changes the pixel value of the complete target object position in the mask to 1, and the pixel value of the remaining positions to 0, and then multiplies it with the original image. In this way, only the single target image is retained in the original image, and the remaining positions are all black. The fused image only has one complete target object, thus completing the single target image segmentation.

[0048] The present invention will be further described below in conjunction with specific embodiments.

[0049] Example

[0050] In general, the single target segmentation solution first needs to train the target detection model, then the target detection model locates each target in the image, locates each target according to the position information obtained, and draws the detection box. Then each detection box is input into the image segmentation model as the ROI area to obtain the segmentation result of the single target.

[0051] The present invention does not require cumbersome steps, and first only needs to determine the ROI area. There are many ways to determine the ROI area. Generally, the approximate ROI area can be determined according to the position of the image where the target is located when the camera is shooting. Then the determined ROI area is input into the improved U-net network, and the network will obtain a mask of a single target. The mask can be merged with the original image to segment the single target. There are also many methods for fusion. Two fusion methods are proposed here: the first fusion can change the pixel value of the single target object in the mask to 1, and the remaining pixel values ​​to 0, and then multiply it with the original image, so that the pixel value of the single target object in the original image remains unchanged, and the pixel value at the remaining position becomes 0, that is, black, so that the image fusion is completed; the second method is to directly cut at the corresponding position of the original image according to the position information of the single target object in the obtained mask, and segment the single target. The improved U-net network of the present invention can not only be used for single target object segmentation, but also can segment all targets when there are multiple targets of the same type in the same image. There is no need to select the ROI area during segmentation, and the image can be directly input into the model, and the segmentation efficiency is much faster than that of the traditional U-net network. Because the structure of the decoder part is simplified, the efficiency of model segmentation will be improved.

[0052] In the embodiment of the present invention, there are two implementation schemes for reducing the number of convolution kernels in the U-net network decoder. One is to reduce the number of 3*3 convolution kernels in each decoding to four times the original number when building the upsampling network structure when building the U-net network by yourself, as shown in the attached figure. Figure 2 As shown in the figure, the number of convolution kernels changes from [256, 128, 64, 32, 16] to [64, 32, 16, 8, 4], and the rest of the U-net network structure remains unchanged, thus achieving the purpose of segmenting a single target. One way is to set the parameter decoder_filters = [64, 32, 16, 8, 4] when calling the encapsulated U-net network in deep learning frameworks such as tensorflow and Keras, which achieves the effect of reducing the number of decoder convolution kernels. Then the backbone selects the shallower mobilenet network, and other parameters during model training are not restricted and are adjusted accordingly according to factors such as the actual target size. After the model is trained, single target object segmentation can be performed.

[0053] In the description of the present invention, unless otherwise specified, "plurality" means two or more than two; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0054] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0055] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A single target object segmentation method based on deep learning, characterized in that: The single target object segmentation method based on deep learning includes: By using an improved U-Net neural network, when an incomplete target and a complete target of the same type coexist, the complete target is segmented out and the incomplete target is filtered out; when the incomplete target and the complete target of the same type coexist, the complete target and the incomplete target are the same object; when segmenting the complete target, the target image has and only has one complete target, and the ROI area is extracted before segmentation; the extracted ROI area only contains one complete target, and there is no restriction on the incomplete target contained; the improved U-net network includes: a plurality of decoders with reduced numbers of partial convolution kernels, and the number of convolution kernels of each decoder is reduced to 4 times of the original; the reduced decoder reduces the features obtained by the model, so that the model does not pay attention to the detailed features of the target, overfits the complete target, only segments the complete target in the ROI area, and ignores the incomplete target.

2. The single target object segmentation method based on deep learning according to claim 1, characterized in that: The ROI region extraction method includes roughly judging the size of the region containing the complete target according to the target center, judging the size of the ROI region according to the target outline, and the ROI region includes multiple incomplete targets; Then the ROI area image is input into the improved U-net network.

3. The single target object segmentation method based on deep learning according to claim 1, characterized in that: The complete target segmentation method specifically includes: (1) Perform image acquisition; (2) The collected image is used to select the ROI region according to the location of the target, and the ROI region image is input into the improved U-Net image segmentation network to segment the image mask of a single target; (3) The single target image mask is fused with the original image, and the fused image is the single target segmentation image.

4. The single target object segmentation method based on deep learning as claimed in claim 3, characterized in that: The fusion method in step (3) includes: changing the pixel value of the complete target object position in the mask to 1, and the pixel values ​​of the remaining positions to 0, and then multiplying with the original image, so that only the single target image is retained in the original image, and the remaining positions are all changed in color; the fused image only has one complete target object, and the single target image segmentation is completed.

5. An improved U-Net neural network for implementing the single target object segmentation method based on deep learning as described in any one of claims 1 to 4.

6. A motor retaining spring groove height detection device that implements the single target object segmentation method based on deep learning as described in any one of claims 1 to 4.

Images