An object boundary recognition method, device and equipment

Abstract

The application relates to an object boundary recognition method, device and equipment, which comprises the following steps: recognizing a target object in an input image to obtain a target object image; filling the target object image into a square, performing binary image segmentation on the target object image to obtain a binary image; performing full-image boundary processing on the input image to determine a full-image boundary; connecting intersection points obtained by intersecting the full-image boundary and the boundary of the binary image to obtain the boundary of the target object. The application can be more accurate in picture boundary prediction, and manual secondary modification is easy to operate because the prediction result can be ensured to be more accurate, and the inaccurate part is convenient for manual boundary supplement.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an object boundary recognition method, device and equipment. BACKGROUND

[0002] With the wide application of supervised learning, the demand for data is increasing, and image data labeling is a big gap. At present, many labeling companies mainly use the combination of preloading and manual labeling when processing image data labeling. However, due to the limitations of algorithms and the poor matching of algorithm processing results with manual secondary labeling, the preloading step often cannot play its full role.

[0003] At present, the commonly used instance segmentation algorithm cannot meet the fine labeling requirements, so manual secondary modification of the algorithm recognition result is needed. Because the existing algorithm cannot control many details, it increases the workload of manual work, and sometimes even consumes more time than without preloading. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides an object boundary recognition method, device and equipment.

[0005] In a first aspect, the present application provides an object boundary recognition method, comprising:

[0006] recognizing a target object in an input image to obtain a target object image;

[0007] filling the target object image into a square, performing binary image segmentation on the target object image to obtain a binary image;

[0008] performing full-image boundary processing on the input image to determine a full-image boundary;

[0009] connecting intersection points obtained by taking the intersection of the full-image boundary and the boundary of the binary image to obtain the boundary of the target object.

[0010] In combination with the first aspect, in a first possible implementation manner of the first aspect, the filling of the target object image into a square comprises:

[0011] performing picture padding on the target object image according to the length of the long side to obtain a square target object image.

[0012] In combination with the first aspect, in a second possible implementation manner of the first aspect, the performing of binary image segmentation on the target object image to obtain a binary image comprises:

[0013] determining a plurality of feature regions of the target object according to the target object image.

[0014] determining a feature threshold corresponding to each of the feature regions;

[0015] performing multi-scale binary image segmentation on the target object image based on the correspondence between the feature regions and the feature threshold by using an image semantic segmentation model to obtain a binary image.

[0016] With reference to the first aspect, in a third possible implementation of the first aspect, the full-image boundary processing on the input image comprises:

[0017] For each pixel point, the pixel values of the surrounding eight pixel points are calculated according to a preset color jump calculation formula to obtain a color jump value;

[0018] The average value of the color jump values of the plurality of pixel points is determined as a jump threshold value;

[0019] The pixel points with color jump values greater than the jump threshold value are determined as color jump points, and the pixel points with color jump values less than the jump threshold value are determined as non-color jump points;

[0020] The full-image boundary is determined based on the color jump points.

[0021] In a second aspect, the present application provides an object boundary recognition device, comprising:

[0022] an identification module configured to identify a target object in an input image to obtain a target object image;

[0023] a segmentation module configured to fill the target object image into a square, perform binary image segmentation on the target object image, and obtain a binary image;

[0024] a processing module configured to perform full-image boundary processing on the input image to determine a full-image boundary;

[0025] an intersection module configured to connect intersection points obtained by taking the intersection of the full-image boundary and the boundary of the binary image to obtain the boundary of the target object.

[0026] With reference to the second aspect, in a first possible implementation of the second aspect, the segmentation module comprises:

[0027] the target object image is filled according to the length of the long side to obtain a square target object image.

[0028] With reference to the second aspect, in a second possible implementation of the second aspect, the segmentation module comprises:

[0029] a first determination unit configured to determine a plurality of feature regions of the target object according to the target object image;

[0030] a second determining unit, configured to determine a feature threshold corresponding to each feature region;

[0031] a segmentation unit, configured to perform multi-scale binary image segmentation on the target object image based on a correspondence between the feature region and the feature threshold by using an image semantic segmentation model, to obtain a binary image.

[0032] With reference to the second aspect, in a third possible implementation of the second aspect, the processing module comprises:

[0033] a calculation unit, configured to calculate, for each pixel point, a color jump value of the pixel point according to a preset color jump calculation formula based on pixel values of eight surrounding pixel points;

[0034] a third determining unit, configured to determine an average value of the color jump values of the plurality of pixel points as a jump threshold;

[0035] a fourth determining unit, configured to determine, as a color jump point, a pixel point whose color jump value is greater than the jump threshold, and determine, as a non-color jump point, a pixel point whose color jump value is less than the jump threshold;

[0036] a fifth determining unit, configured to determine a full-image boundary based on the color jump point.

[0037] In a third aspect, the present application provides an object boundary recognition device, comprising a memory and a processor, the memory stores a computer program executable on the processor, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0038] In a fourth aspect, the present application provides a computer readable medium having non-volatile program code executable by a processor, the program code causing the processor to execute the method of the first aspect.

[0039] The above technical solution provided by the embodiments of the present application has the following advantages compared with the prior art:

[0040] The method provided by the embodiments of the present application is not only more accurate in picture boundary prediction, but also easier to operate when a person makes secondary modification thereon, because the prediction result is more accurate and the inaccurate part is convenient for manual boundary filling. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, for those of ordinary skill in the art, the other accompanying drawings can be obtained based on these accompanying drawings without any creative effort.

[0043] Figure 1 A flowchart of an object boundary recognition method provided by the embodiment of the present application;

[0044] Figure 2 A flowchart of step S102 provided by the embodiment of the present application;

[0045] Figure 3 A vehicle picture output when the confidence is 0.7 provided by the embodiment of the present application;

[0046] Figure 4 A vehicle picture output when the confidence is 0.8 provided by the embodiment of the present application;

[0047] Figure 5 A flowchart of step S103 provided by the embodiment of the present application;

[0048] Figure 6 An exemplary principle explanation picture when a full picture boundary processing is provided by the embodiment of the present application;

[0049] Figure 7 A picture obtained after a full picture boundary processing provided by the embodiment of the present application;

[0050] Figure 8 Another exemplary principle explanation picture when a full picture boundary processing is provided by the embodiment of the present application;

[0051] Figure 9 Another exemplary principle explanation picture when a full picture boundary processing is provided by the embodiment of the present application;

[0052] Figure 10 A structure diagram of an object boundary recognition device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art without any creative effort belong to the scope of protection of the present application.

[0054] At present, the commonly used instance segmentation algorithm cannot meet the fine degree requirement of fine annotation, so manual secondary modification is needed for the algorithm recognition result. Because the existing algorithm cannot control many details, it increases a lot of workload for manual work, and sometimes even consumes more time than not preloading. Therefore, in an embodiment of the present application, an object boundary recognition method is provided, as shown in Figure 1 The method comprises the following steps:

[0055] In step S101, a target object in an input image is recognized to obtain a target object image.

[0056] In this step, the object recognition is performed by using a target detection algorithm CascadeR-CNN (CVPR2018), and a rectangular frame is obtained. The algorithm CascadeR-CNN (CVPR2018) is described in the following website:

[0057] http: / / openaccess.thecvf.com / content_cvpr_2018 / papers / Cai_Cascade_R-CNN_Delving_CVPR_2018_paper.pdf;

[0058] When the target object is recognized by using the target detection algorithm, the object features of the target object are also detected to obtain a feature region corresponding to each object feature.

[0059] The explanation of the object features can be found in the following link:

[0060] https: / / blog.csdn.net / dengheCSDN / article / details / 77848246, that is, the features can be understood as the projection of different features of an image obtained by filtering and deconvolving the image

[0061] In step S102, the target object image is filled into a square, and binary image segmentation is performed on the target object image to obtain a binary image.

[0062] In this step, the rectangular frame is subjected to picture padding processing, and the rectangular size is m*n. According to the larger one of m and n, the square is filled with black color. The purpose of filling the square is to prevent the deformation of the object in the rectangular frame after resizing. The picture of the square is only different in definition after resizing, and does not affect the object ratio. For example, a typical standing person picture is taken as an example. Because the image segmentation must resize the square, the standing person picture will be resized to be very fat, and some details of the person will be lost.

[0063] The picture padding processing is performed because people generally directly throw the rectangular frame into CNN training, but the padding processing can make the effect better due to the particularity of the square.

[0064] As shown in Figure 2 The step S102 of performing the binary image segmentation on the target object image to obtain the binary image can include the following steps:

[0065] In step S201, a plurality of feature regions of the target object are determined according to the target object image.

[0066] In step S202, a feature threshold corresponding to each feature region is determined.

[0067] In step S203, based on the correspondence between the feature region and the feature threshold, a multi-scale binary image segmentation is performed on the target object image by using an image semantic segmentation model to obtain a binary image.

[0068] The region threshold is set. When the confidence of the last layer of the binary image segmentation is output, each algorithm uniformly sets the threshold to obtain the binary image. After a large number of experiments, targeted adjustment is adopted. When identifying, different thresholds are set for different regions, so that the output result is more conducive to manual secondary labeling, and the algorithm plays a greater role. After some experiments, it is found that when the global threshold is set to 0.5, 0.6, 0.7, 0.8 and 0.9, the effect is best when the threshold is set to 0.7 and 0.8. By comparing the subtle differences, it is found that for a car, the upper and lower proportions are 1:1, that is, the car picture is cut into two equal parts, and the upper and lower parts are set to different thresholds. When the upper threshold is set to 0.7, the upper part of the car is relatively smooth and the boundary is relatively obvious. When the lower threshold is set to 0.8, the boundary between the wheels and chassis of the lower part of the car and the ground is not obvious.

[0069] The algorithm DeepLabV3 binary image segmentation is performed. The algorithm DeepLabV3 is referred to the website https: / / github.com / sthalles / deeplab_v3. The purpose is to output the confidence picture before setting the global threshold. Taking a vehicle picture as an example, Figure 3 and Figure 4 is the picture output when the confidence is a number between 0 and 1. When the threshold is set to 0.7 or 0.8, the confidence greater than 0.7 or 0.8 becomes 1, and vice versa. 0 represents non-car, and 1 represents car. The present application can give more accurate threshold according to different situations.

[0070] In step S103, the whole image boundary is determined by performing the whole image boundary processing on the input image.

[0071] As Figure 5 shown in the figure, the step S103 can include the following steps:

[0072] Step S301, for each pixel point, the pixel value of the surrounding eight pixel points is calculated according to the preset color jump calculation formula to calculate the color jump value;

[0073] For example, the preset color jump calculation formula is as follows:

[0074] dis=abs(a(i,j)-a(i+1,j))+abs(a(i,j)-a(i-1,j))+abs(a(i,j)-a(i,j+1))+abs(a(i,j)-a(i,j-1))+abs(a(i,j)-a(i+1,j+1))+abs(a(i,j)-a(i-1,j+1))+abs(a(i,j)-a(i+1,j-1))+abs(a(i,j)-a(i-1,j-1));

[0075] Wherein, Dis is a color jump parameter of a point, which is used to determine whether it is a color jump point, Abs() represents the absolute value, A(i,j) represents the pixel value (0-255) of the position (i,j), and the distribution of A(i,j) parameter is to remove a circle of points of the image boundary and only consider (i-2)*(j-2) pixel points (see each pixel point in the thick black square frame line in the figure Figure 6 ), the dis parameter of each pixel position is calculated to form an array.

[0076] As Figure 6 shown, for example, an 8*8 picture can form an array of 6*6=36 elements (the edge position is not considered), and the obtained picture is as shown in the figure Figure 7 . Figure 7 The pixel value of each pixel point in the full image is a number between 0 and 255.

[0077] However, as Figure 8 shown, assuming Figure 8 the pixel value of each pixel point in area A is 1, and the pixel value of each pixel point in area B is 0, after the full image boundary processing, the obtained picture only remains the black wavy line containing the middle (as shown in the figure) Figure 9 , and the other positions are ordinary non-jump points (non-jump points refer to the pixel values of the eight pixel points around the non-jump point being the same or similar to the pixel value of the non-jump point), the position of color change can be extracted.

[0078] Step S302, the average value of the color jump values corresponding to the plurality of pixel points is determined as the jump threshold value;

[0079] The region threshold is set, and a uniform threshold is set without considering the difference in confidence. The present application compares a large number of confidence maps, and obtains that different objects can set the threshold according to the characteristics of the objects.

[0080] In step S303, the pixel point with the color jump value greater than the jump threshold is determined as a color jump point, and the pixel point with the color jump value less than the jump threshold is determined as a non-color jump point.

[0081] The pixel point with the color jump value greater than the jump threshold is determined as a color jump point, and the pixel point with the color jump value less than the jump threshold is determined as a non-color jump point. Figure 7 .

[0082] In step S304, the full-image boundary is determined based on the color jump point.

[0083] The full-image boundary is also the mask image boundary.

[0084] In step S104, the intersection point obtained by taking the intersection of the full-image boundary and the boundary of the binary image is connected to obtain the boundary of the target object.

[0085] Since the full-image boundary certainly contains the real boundary, the intersection of the two can select the boundary points in the mask image that are very accurate and exclude the boundary points in the mask image that are not accurate. Then, the intersection points are connected by a straight line to obtain the final prediction result. The result obtained is not only more accurate in picture boundary prediction, but also easier for manual secondary modification, because the prediction result is more accurate and the inaccurate part is convenient for manual boundary modification. According to the actual test results, compared with pure manual labeling, the time saving is more than 60%.

[0086] In another embodiment of the present application, an object boundary identification device is also provided, as shown in Figure 10 The device comprises:

[0087] An identification module 11 is configured to identify a target object in an input image to obtain a target object image.

[0088] A segmentation module 12 is configured to fill the target object image into a square, perform binary image segmentation on the target object image, and obtain a binary image.

[0089] A processing module 13 is configured to perform full-image boundary processing on the input image to determine a full-image boundary.

[0090] The intersection module 14 is configured to connect intersection points obtained by intersecting the full image boundary with a boundary of the binary image to obtain a boundary of the target object.

[0091] In another embodiment of the present application, the segmentation module comprises:

[0092] The target object image is filled according to the length of the long side to obtain a square target object image.

[0093] In another embodiment of the present application, the segmentation module comprises:

[0094] The first determination unit is configured to determine a plurality of feature regions of the target object according to the target object image.

[0095] The second determination unit is configured to determine a feature threshold corresponding to each feature region.

[0096] The segmentation unit is configured to perform multi-scale binary image segmentation on the target object image by using an image semantic segmentation model based on a corresponding relationship between the feature regions and the feature thresholds to obtain a binary image.

[0097] In another embodiment of the present application, the processing module comprises:

[0098] The calculation unit is configured to calculate, for each pixel point, a color jump value of the pixel point according to a preset color jump calculation formula.

[0099] The third determination unit is configured to determine an average value of the color jump values of the plurality of pixel points as a jump threshold.

[0100] The fourth determination unit is configured to determine, as a color jump point, a pixel point with a color jump value greater than the jump threshold, and determine, as a non-color jump point, a pixel point with a color jump value less than the jump threshold.

[0101] The fifth determination unit is configured to determine a full image boundary based on the color jump point.

[0102] In another embodiment of the present application, an object boundary recognition device is also provided, which comprises a memory and a processor, the memory stores a computer program executable on the processor, and the processor implements the steps of the method according to the above method embodiments when executing the computer program.

[0103] In another embodiment of the present application, a computer readable medium having non-volatile program code executable by a processor is also provided, and the program code causes the processor to execute the method according to the method embodiments.

[0104] It has to be noted that, in the present document, relational terms are intended only to convey a possible relationship between elements or

[0105] The above description is merely that of a specific implementation to enable a person skilled in the art to understand or implement the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An object boundary identification method characterized by, The method comprises the following steps: recognize a target object in an input image to obtain a target object image; fill the target object image into a square, perform picture padding on the target object image according to the length of the long side to obtain a square target object image, and perform binary image segmentation on the target object image to obtain a binary image; perform full-image boundary processing on the input image to determine a full-image boundary; comprising: for each pixel point, calculate the color jump value of the pixel value of the surrounding eight pixel points according to a preset color jump calculation formula, wherein the color jump value is the sum of the absolute values of the difference between the pixel value of the pixel point to be calculated and the pixel value of each of the surrounding eight pixel points, and the pixel point to be calculated is any pixel point in the input image except the outermost circle of pixel points, and the edge position is not considered; determine the average value of the color jump values of a plurality of pixel points as a jump threshold value; determine the pixel points with color jump values greater than the jump threshold value as color jump points, and determine the pixel points with color jump values less than the jump threshold value as non-color jump points; determine the full-image boundary based on the color jump points; connect the intersection points obtained by taking the intersection of the full-image boundary and the boundary of the binary image to obtain the boundary of the target object.

2. The object boundary identification method according to claim 1, characterized in that, perform binary image segmentation on the target object image to obtain a binary image, comprising: determine a plurality of feature regions of the target object according to the target object image; determine a feature threshold value corresponding to each feature region; based on the correspondence between the feature regions and the feature threshold values, perform multi-scale binary image segmentation on the target object image using an image semantic segmentation model to obtain a binary image.

3. An object boundary identification apparatus, characterized by, The method comprises the following steps: a recognition module for recognizing a target object in an input image to obtain a target object image; a segmentation module for filling the target object image into a square, performing binary image segmentation on the target object image to obtain a binary image; a processing module for performing full-image boundary processing on the input image to determine a full-image boundary; the processing module comprises: a calculation unit for calculating the color jump value of the pixel value of the surrounding eight pixel points according to a preset color jump calculation formula for each pixel point, wherein the color jump value is the sum of the absolute values of the difference between the pixel value of the pixel point to be calculated and the pixel value of each of the surrounding eight pixel points, and the pixel point to be calculated is any pixel point in the input image except the outermost circle of pixel points, and the edge position is not considered; a third determination unit for determining the average value of the color jump values of a plurality of pixel points as a jump threshold value; a fourth determination unit for determining the pixel points with color jump values greater than the jump threshold value as color jump points, and determining the pixel points with color jump values less than the jump threshold value as non-color jump points; a fifth determination unit for determining the full-image boundary based on the color jump points; an intersection taking module for connecting the intersection points obtained by taking the intersection of the full-image boundary and the boundary of the binary image to obtain the boundary of the target object.

4. The object boundary identification apparatus according to claim 3, characterized by the segmentation module comprises: The target object image is picture filled according to the length of the long side to obtain a square target object image.

5. The object boundary identification apparatus according to claim 3, wherein The segmentation module comprises: A first determination unit configured to determine a plurality of feature regions of the target object according to the target object image; A second determination unit configured to determine a feature threshold corresponding to each of the feature regions; A segmentation unit configured to perform multi-scale binary image segmentation on the target object image by using an image semantic segmentation model based on the correspondence between the feature regions and the feature threshold to obtain a binary image.

6. An object boundary identification apparatus comprising a memory, a processor, the memory having stored therein a computer program executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-2.

7. A computer readable medium having a non-transitory program code executable by a processor, the program code comprising instructions for: The program code causes the processor to execute the method of any one of claims 1-2.

Images