Transparent object extraction method and device

Abstract

The invention provides a transparent object extraction method and device, and the method comprises the steps: carrying out the edge extraction of a gray image of a to-be-segmented image, and determining a first region representing a to-be-segmented object and a second region representing a background in the to-be-segmented image based on an extraction result; determining an HSV mask of the to-be-segmented image based on the gray information of the first region and the second region, and segmenting a fourth region in an HSV space based on the HSV mask to obtain a third region representing a to-be-segmented object, the fourth region representing a region of the to-be-segmented image after the edge of the to-be-segmented image is removed; and jointly applying a region growing algorithm to the first region and the third region to obtain a target image of the to-be-segmented image. According to the invention, through combination of the grey-scale map and the HSV space, unique information of the transparent object facing illumination is extracted and is separated from the background.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a method and device for extracting a transparent object. Background Art

[0002] Due to their own optical properties, transparent objects are difficult to distinguish from the background and other objects, making them difficult to use in situations that require fast response characteristics, such as obstacle avoidance detection, transparent container sorting and operation, etc.

[0003] Many researchers use multispectral imaging to improve the contrast between transparent objects and the background by using light sources of different wavelengths, but this requires professional equipment; scholars use deep learning technology to learn the characteristics of transparent objects and improve the accuracy of transparent object extraction, but it requires a large number of samples. For this purpose, different perspectives and lighting conditions are used for image synthesis to enhance the diversity of data, but data annotation is time-consuming; experts pre-process images to improve the separation between transparent objects and the background, but the separation values ​​need to be manually fine-tuned for different types of objects and cannot be updated in real time. In recent years, some scholars have used grayscale images to distinguish the brightness values ​​of transparent objects from the background under illumination, thereby separating the background, but due to the singleness of the separation conditions and the strict requirements for lighting, they cannot adapt to the loss of contours caused by low contrast and blurred edge information.

[0004] Therefore, how to quickly and accurately extract transparent objects remains a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a transparent object extraction method and device, which are used to solve the defect that it is difficult to accurately extract transparent objects in the prior art, and realize a rapid and accurate transparent object extraction method.

[0006] The present invention provides a transparent object extraction method, comprising: Performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; Determine an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segment in a fourth region of the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; A region growing algorithm is applied to the first region and the third region together to obtain a target image of the image to be segmented.

[0007] According to a transparent object extraction method provided by the present invention, the grayscale information includes a grayscale peak value and a grayscale average value, and the step of determining the HSV mask of the image to be segmented based on the grayscale information of the first area and the second area specifically includes: Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peaks of the first area and the second area; In the case where the object to be segmented is a completely transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the first area; In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

[0008] According to a transparent object extraction method provided by the present invention, the step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first area and the second area specifically includes: When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object; Otherwise, it is determined that the object to be segmented is a semi-transparent object.

[0009] According to a transparent object extraction method provided by the present invention, the region growing criterion of the region growing algorithm includes a texture similarity criterion, an HSV value similarity criterion and a pixel distance criterion.

[0010] According to a transparent object extraction method provided by the present invention, the step of applying a region growing algorithm to the first region and the third region to obtain a target image of the image to be segmented specifically includes: Convert the first region to the HSV space, perform an opening operation on the first region, and perform a convolution based on the HSV mask, and during the convolution process, spread a first preset number of seed points as initial seed points of the part according to a preset spreading method; After determining the third area, performing convolution based on the HSV mask on the third area, and during the convolution process, sowing a second preset number of seed points as initial seed points according to the preset sowing method; Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

[0011] According to a transparent object extraction method provided by the present invention, the preset spreading mode is: random spreading in the first area and the third area respectively.

[0012] According to a transparent object extraction method provided by the present invention, the step of implementing region expansion based on the initial seed point and the region growth criterion specifically includes: Based on the initial seed point in the first area, implementing area expansion according to the pixel distance criterion and the texture similarity criterion; In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion and the HSV value similarity criterion.

[0013] The present invention also provides a transparent object extraction device, comprising: A preliminary segmentation module, used for performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; An HSV segmentation module, configured to determine an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; The extraction module is used to apply a region growing algorithm to the first region and the third region to obtain a target image of the image to be segmented.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned transparent object extraction methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the transparent object extraction method described in any one of the above methods is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the transparent object extraction method described above is implemented.

[0017] The transparent object extraction method and device provided by the present invention, by combining the grayscale image and the HSV space, extract the unique information of the transparent object when facing light and separate it from the background, to obtain the third area representing the object, and apply the region growing algorithm to the first area and the third area to obtain the target image representing the image to be segmented. It can cope with different lighting conditions and complex background information and improve the integrity of the transparent object segmentation area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the transparent object extraction method provided by the present invention; Figure 2 is a grayscale image of the image to be segmented in the transparent object extraction method provided by the present invention; Figure 3 It is a rough outline map of the image to be segmented in the transparent object extraction method provided by the present invention; Figure 4 It is a grayscale peak image corresponding to a completely transparent object and a background in the transparent object extraction method provided by the present invention; Figure 5 It is a grayscale peak image corresponding to the semi-transparent object and the background in the transparent object extraction method provided by the present invention; Figure 6 is the HSV intensity histogram of the image to be segmented in the transparent object extraction method provided by the present invention; Figure 7 is an HSV intensity histogram of the first area corresponding to the completely transparent object in the transparent object extraction method provided by the present invention; Figure 8 is an HSV intensity histogram of the first area corresponding to the semi-transparent object in the transparent object extraction method provided by the present invention; Fig. 9 is an HSV intensity histogram of the second area in the transparent object extraction method provided by the present invention; Fig.10 It is a schematic diagram after the first area is opened and operated in the transparent object extraction method provided by the present invention; Fig.11 It is a schematic diagram of the change in the number of initial seed points in the transparent object extraction method provided by the present invention; Fig.12 It is a schematic diagram of the change of the initial seed point sowing mode in the transparent object extraction method provided by the present invention; Fig.13 It is a schematic diagram of the growth result of the contour area of ​​the object to be segmented in the transparent object extraction method provided by the present invention; Fig.14 It is a schematic diagram of the growth result of the internal region of the object to be segmented in the transparent object extraction method provided by the present invention; Fig.15is a schematic diagram of a target area extracted by the transparent object extraction method provided by the present invention; Fig.16 This is the second flow chart of the transparent object extraction method provided by the present invention; Fig.17 It is a structural schematic diagram of the transparent object extraction device provided by the present invention; Fig.18 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figures 1 to 16 The transparent object extraction method of the present invention is introduced as follows: Figure 1 As shown, including: Step 101, performing edge extraction on a grayscale image of an image to be segmented, and determining a first region representing an object to be segmented and a second region representing a background in the image to be segmented based on the extraction result; When transparent objects are used as detection objects, facing complex lighting conditions and changing backgrounds, due to the inconsistency of lighting, it is difficult to separate the transparent objects from the background in the detection image, making the shape of the segmented transparent objects incomplete.

[0022] To this end, the present invention introduces HSV (Hue, Saturation, Value) information into the transparent object segmentation method, in order to achieve more accurate image segmentation based on the grayscale image, so as to more accurately extract transparent objects.

[0023] To achieve image segmentation based on HSV information, it is necessary to determine a suitable HSV mask to screen the hue, saturation and brightness of the pixels, and consider the pixels within the range of the HSV mask as the segmentation results. It can be understood that the range of the HSV mask, that is, the upper and lower limits of the HSV mask, has a great influence on the image segmentation results based on HSV.

[0024] Since HSV information contains information of three channels, namely hue, saturation and brightness, when the mask range of one channel is adjusted during the process of selecting a suitable HSV mask, the ranges of the other two channels will also change, making it difficult to determine the range of the HSV mask.

[0025] Therefore, in order to achieve the transparent object extraction task based on grayscale information and HSV information, the present invention determines a suitable HSV mask in combination with the grayscale information of the object and the background in the image to be segmented.

[0026] The image to be segmented may be a color image obtained by various types of photographing equipment and containing a transparent object to be identified.

[0027] Specifically, the image to be segmented is converted into a grayscale image, such as Figure 2 As shown, an edge detection algorithm is applied to the grayscale image of the image to be segmented to realize edge extraction of the object to be segmented.

[0028] Grayscale images mainly highlight areas with large brightness changes. Since the translucent material itself will reflect, refract and transmit light, there will be a brightness difference between its brightness and the light reflected by the background itself. This difference can be used to extract contours and distinguish between objects and backgrounds.

[0029] Optionally, the canny algorithm is used as the edge detection algorithm. Specifically, a 5×5 Gaussian kernel is used to perform Gaussian filtering on the image to be segmented to remove noise, and then the image gradient is calculated: ; ; ; ; In the formula, l Represents the gray value of the pixel of the image to be segmented, G x and G y Represent the horizontal and vertical gradients respectively, θ is the gradient direction, and the position where the gray value changes greatly is found by calculating the gradient as the identified edge.

[0030] On this basis, maximum suppression is performed, edge preservation is performed with double thresholds, and finally the edges are connected to obtain the rough outline of the object to be segmented, such as Figure 3 shown.

[0031] The image to be segmented is divided based on the rough contour to be segmented, and a first region representing the object to be segmented and a second region representing the background are determined.

[0032] It can be understood that the division based on the rough contour is a rough division step, so that the divided first area can roughly cover the object area. Figure 3 As an example, the edge extraction result can be determined to completely surround Figure 3For a regular polygon with a central edge, the area inside the regular polygon is regarded as the first area representing the object, and the area outside the regular polygon is regarded as the second area representing the background.

[0033] Step 102, determining an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segmenting the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented; Optionally, the grayscale information represents information extracted from the grayscale image, and may include a grayscale peak value and a grayscale average value, etc.

[0034] On this basis, the grayscale information of the first region represents the characteristics of the object to be segmented in the grayscale space, and the grayscale information of the second region represents the characteristics of the background in the grayscale space.

[0035] The grayscale information of the first area is compared with the grayscale information of the second information, and the HSV mask used in HSV space segmentation is determined according to the comparison result, that is, a suitable HSV mask is determined based on the characteristics of the object to be segmented and the background in the grayscale space to segment the fourth area in the HSV space.

[0036] The fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented.

[0037] In a feasible implementation, convolution is performed on the first region, a portion of the first region representing an edge of the object to be segmented is extracted as the fifth region, and a difference between the image to be segmented and the fifth region is taken as the fourth region.

[0038] On this basis, the third region is the result of segmenting the fourth region using the HSV mask, which can be understood as representing the region other than the edge of the object.

[0039] Step 103: Apply a region growing algorithm to the first region and the third region together to obtain a target image of the image to be segmented.

[0040] The region growing algorithm is a method for image segmentation based on pixel similarity. The basic idea is to start from one or more seed points and merge adjacent pixels into the same region according to similarity criteria (such as grayscale value or color value) until there are no adjacent pixels that meet the criteria.

[0041] It can be understood that, through the above processing method, the first region is the region that characterizes the outline of the object, and the third region can be considered to be the region that roughly characterizes the interior of the object. Therefore, by applying the regional growing algorithm to the first region and the third region respectively, and merging their respective expansion results, the target area corresponding to the complete object can be obtained.

[0042] Optionally, initial seed points are sown in the first region and the third region respectively, and respective regional growth criteria are determined, and region expansion is performed based on the initial seed points. In the absence of adjacent pixels that meet the criteria, the result of regional growth is obtained, and the result is used as the final extracted target image representing the object to be segmented.

[0043] It is understandable that since the third area is the area representing the object to be segmented based on the HSV mask in the HSV space, and the HSV space highlights the color, color saturation and brightness, a single grayscale image cannot provide background information of similar colors, where color saturation and brightness will differ due to the incident angle of light. At the same time, due to the influence of transparent objects on light intensity, the brightness of different positions of the transparent objects is different, which differs from the uniform brightness and color of the background.

[0044] Therefore, it can be considered that, compared with the common contour division based on grayscale value, the third region combines the HSV information of the image to be segmented, thereby obtaining a more accurate object region segmentation result in the task of extracting transparent objects. On this basis, region growing is performed based on the third region, and a more accurate decomposition result that characterizes the contour of the object to be segmented can be obtained, and the object to be segmented can be better separated from the background.

[0045] The present invention combines the grayscale image and the HSV space to extract the unique information of the transparent object when facing light and separate it from the background, so as to obtain the third area representing the object. The region growing algorithm is applied to the first area and the third area to obtain the target image representing the image to be segmented. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented area of ​​the transparent object.

[0046] In the transparent object extraction method of the present invention, the grayscale information includes a grayscale peak value and a grayscale average value, and the step of determining the HSV mask of the image to be segmented based on the grayscale information of the first area and the second area specifically includes: Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peaks of the first area and the second area; Furthermore, the category of the object to be segmented is determined to be a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first region and the second region.

[0047] Specifically, there is a difference in the grayscale peaks between the background and the object, which is caused by the reflection intensity of the material to light. Usually, e.g. Figure 4 and Figure 5 As shown, for a completely transparent object, the grayscale peak value of the object is greater than the grayscale peak value of the background; for a semi-transparent object, the grayscale peak value of the object is less than the grayscale peak value of the background.

[0048] Based on this, the grayscale peak values ​​of the first area and the second area can be used to determine whether the object to be segmented is a completely transparent object or a semi-transparent object.

[0049] On this basis, the HSV mask corresponding to the object to be segmented is determined according to the category of the object to be segmented. The HSV intensity histograms of the image to be segmented, the first area corresponding to the completely transparent object, the first area corresponding to the semi-transparent object, and the second area corresponding to the background are respectively as follows: Figure 6 , Figure 7 , Figure 8 and Fig. 9 shown.

[0050] In the case where the object to be segmented is a completely transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the first area; When the object to be segmented is a completely transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first region.

[0051] Specifically, in the grayscale image of the object to be segmented, the coordinates of all pixel points corresponding to the grayscale peak values ​​of the first region are determined and recorded as grayscale coordinates.

[0052] The grayscale coordinates are mapped back to the original image to be segmented, and the pixels corresponding to the grayscale coordinates in the image to be segmented are obtained and organized into a first pixel set.

[0053] Obtain an intensity distribution histogram corresponding to H, S, and V of each pixel in the first pixel set.

[0054] It can be understood that since the pixels in the first pixel set are all pixel points corresponding to the grayscale peak value of the first region, the H values, S values ​​and V values ​​of the pixels in the first pixel set are all concentratedly distributed, and the median, mode or mean of the H value, S value and V value can be taken as the HSV value of the first pixel set, and the determined HSV value can be used as the upper limit of the HSV mask.

[0055] The method of determining the lower limit of the HSV mask based on the grayscale average value of the first area is similar. The grayscale average value of the first area is calculated, and the coordinates of the grayscale average value of the first area in the grayscale image are determined. After mapping it back to the image to be segmented, the pixels of the corresponding coordinates are extracted and organized into a second pixel set.

[0056] Obtain the intensity distribution histogram corresponding to H, S, and V of each pixel in the second pixel set, and determine the lower limit of the HSV mask.

[0057] In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

[0058] When the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first region, the grayscale average value of the second region is calculated, and the lower limit of the HSV mask is determined according to the grayscale average value of the second region.

[0059] The upper and lower limits are determined in the same manner as the upper and lower limits of the HSV mask of a completely transparent object, and therefore will not be described in detail.

[0060] It can be understood that by determining the upper and lower limits of the HSV mask, that is, determining the screening range of the HSV mask, in the HSV space corresponding to the first area, the object and the background can be separated by screening using the HSV mask to obtain the third area.

[0061] The present invention provides additional channel information to compensate for the constraint of a single gray value by matching the HSV value of the pixel corresponding to the maximum gray value peak with the HSV value of the original image, thereby greatly improving the segmentation effect.

[0062] In the transparent object extraction method of the present invention, the step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first area and the second area specifically includes: When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object; Otherwise, it is determined that the object to be segmented is a semi-transparent object.

[0063] Determine the grayscale peak value of the first region P i , used to characterize the grayscale peak value of the object to be segmented.

[0064] Determine the grayscale peak of the second region P b , used to characterize the grayscale peak of the background.

[0065] exist P i > P b In the case of , the object to be segmented is considered to be a completely transparent object; otherwise, the object to be segmented is considered to be a semi-transparent object.

[0066] In the transparent object extraction method of the present invention, the region growing criteria of the region growing algorithm include a texture similarity criterion, an HSV value similarity criterion and a pixel point distance criterion.

[0067] In the region growing algorithm, it is necessary to predetermine the region growing criterion to be followed by the region expansion. Since the method of the present invention introduces the characteristics of the HSV space, the corresponding defined region growing criterion also includes the HSV value similarity criterion.

[0068] Optionally, the HSV value similarity between pixels can be expressed by the following formula: ; In the formula, It is a weight that adapts to the different brightness and luminance caused by reflection and refraction under similar colors.

[0069] Will D HSV Two pixels whose values ​​are smaller than the preset HSV threshold are considered to be similar pixels and can be merged.

[0070] In addition, the defined region growing criteria also include texture similarity criteria, which specifically calculates the similarity of local textures through the local binary pattern (LBP) to adapt to complex environmental background information. When the LBP value of the surrounding points and the seed point is less than the set threshold, expansion can be performed.

[0071] In addition, the defined region growth criterion also includes the pixel distance criterion to ensure the connectivity of the internal region of the object, which can be expressed by the following formula: ; In the formula, P is the pixel coordinate.

[0072] It should be noted that, among the three criteria defined in this implementation, one or more of them may be selected for use according to requirements at different locations of area expansion.

[0073] In the transparent object extraction method of the present invention, the step of applying a region growing algorithm to the first region and the third region to obtain the target image of the image to be segmented specifically includes: After performing an opening operation on the contour of the third region, performing convolution based on the HSV mask, and during the convolution process, sowing a first preset number of seed points as initial seed points of the part according to a preset sowing method; Convert the first region to the HSV space, perform an opening operation on the first region, and perform a convolution based on the HSV mask, and during the convolution process, spread a first preset number of seed points as initial seed points of the part according to a preset spreading method; In order to avoid omission of the object region to be segmented during the region growing process, a certain number of initial seed points are sown in the third region representing the interior of the object and the first region representing the contour of the object, respectively, and the region growing operation is completed in these two regions at the same time.

[0074] Specifically, the spreading process of the first region is performed after the first region is determined. Since the first region is obtained based on the grayscale image segmentation of the image to be segmented, the first region needs to be converted into the HSV space first.

[0075] On this basis, the first region in the HSV space is opened to avoid omissions in the process of extracting the edge of the object to be segmented. The first region after the opening operation is as follows Fig.10 shown.

[0076] During the opening operation, a mask convolution operation is performed on it with a size of 5×5 as the first mask convolution, so that a more accurate area for characterizing the edge of the object can be further separated in the first area according to the determined HSV mask.

[0077] And in the process of convolution, a first preset number of seed points are spread as a part of the initial seed points according to a preset spreading method, so that the spreading of the initial seed points of the first area representing the contour of the object is completed.

[0078] After determining the third area, performing convolution based on the HSV mask on the third area, and during the convolution process, sowing a second preset number of seed points according to the preset sowing method as initial seed points of another part; Specifically, the spreading process on the third area is performed after the third area is determined.

[0079] It should be noted that since the third region is the result of segmenting the fourth region, the fourth region is the part of the image to be segmented where the fifth region representing the edge of the object is removed. In the present embodiment, a more accurate object edge is obtained by performing the first convolution on the first region and extracting the HSV information. Therefore, the result of the mask convolution on the first region is directly used as the fifth region, and the remaining part of the image to be segmented after removing the fifth region is used as the fourth region, thereby obtaining the third region based on the fourth region.

[0080] After determining the third area, a 5×5 mask is used to perform a convolution operation on the third area as the second mask convolution, and during the second convolution, a second preset number of seed points are sown in a preset sowing method as initial seed points for another part.

[0081] At this point, the initial seed point sowing of the third area representing the interior of the object is completed.

[0082] The preset spreading method predetermines the spreading position of the initial seed points. For example, it can be completely random spreading or random spreading in each predetermined area, so that the initial seeds are more evenly distributed in the entire area.

[0083] The first preset number and the second preset number, that is, the number of seed points sown twice, can be adjusted according to the specific experimental process.

[0084] In this implementation, the exploration process for the number of initial seed points (n) to be sown is as follows: Fig.11 As shown in the figure, when n is small, the seed points can only grow in the area around themselves and cannot cover the entire object; when n reaches 15, the seed points gather into one area (the bottom of the bowl), and when n reaches 20, the shape and outline of the bowl are clearly displayed. However, the contact area between the seed and the background increases, resulting in incorrect segmentation of the object (n=25). Therefore, the number of initial seed points for both sowings is set to 20.

[0085] Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

[0086] After the initial seed point is determined, the region expansion can be achieved based on the initial seed point and the predefined region growth criterion.

[0087] Optionally, starting from the initial seed point, check whether its 8-neighborhood satisfies the region growing criterion. If so, add the pixel to the current region and continue to expand to other unvisited pixels.

[0088] After the expansion stops, that is, when there are no new pixels that can be expanded, an extended image representing the object based on the third area as the growth basis and an extended image representing the object contour based on the first area as the growth basis are obtained, and the two extended images are merged as the target image of the image to be segmented.

[0089] In the transparent object extraction method of the present invention, the preset spreading mode is: random spreading in the first area and the third area respectively.

[0090] In this embodiment, the exploration process of the preset spreading mode is as follows: Fig.12 As shown, the first area representing the contour of the object and the third area representing the interior of the object are spread respectively. On this basis, the influence of different spreading methods such as random spreading and uniform spreading on the transparent object extraction effect in the contour and interior of the object is further explored.

[0091] Specifically, when uniform seeding processes edges, due to the inconsistent reflection of light by transparent objects, the edges of objects overlap with the background, and random seeds are concentrated in a certain area to produce segmentation. When processing the interior of an object, there is not much difference between the two distributions. Since uniform distribution requires the consistency of light, the reflection and light transmission properties of the material cannot meet this condition.

[0092] Therefore, the final choice is to randomly distribute the initial seeds around the outline and inside of the object.

[0093] In other feasible implementations, different sowing methods may be configured for the initial seeds in the first area and the third area of ​​the object according to actual exploration conditions.

[0094] In the transparent object extraction method of the present invention, the step of implementing region expansion based on the initial seed point and the region growth criterion specifically includes: Based on the initial seed point of the first region, implementing region expansion according to the pixel distance criterion and the texture similarity criterion; In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion and the HSV value similarity criterion.

[0095] It can be understood that, since in the process of spreading the initial seed points, the outline of the object and the interior of the object are distinguished and spread separately, therefore, when the region is expanded, different region growth criteria can be further applied to the outline of the object and the interior of the object to achieve differentiated region expansion.

[0096] Specifically, since the size of the interior of the object and the edge of the contour are different, the growth constraints will be relaxed when processing the contour, and the pixel distance criterion and texture similarity criterion will be taken as the main constraints. That is to say, when processing the contour, the pixels that meet the pixel distance criterion and texture similarity criterion can be merged for expansion.

[0097] When dealing with the interior of an object, the three defined criteria need to be met simultaneously before they can be merged and expanded.

[0098] According to the above processing method, after the region expansion is completed, the target segment corresponding to the outline of the object and the target segment corresponding to the interior of the object are obtained respectively, such as Fig.13 and Fig.14 As shown in , the target region can be obtained by merging the two target segments, such as Fig.15 It should be noted that in order to more clearly show the mentioned regions in the attached figure, black and white drawings are used, and the region growing process should be completed in the original image to be segmented, that is, it should be completed in the image containing HSV information.

[0099] On this basis, a complete process of extracting the target area of ​​the object to be segmented in the image to be segmented is as follows: Fig.16 shown.

[0100] The present invention combines grayscale images and HSV space to extract the unique information of transparent objects when facing light and separate it from the background. In view of the different light transmission intensities of translucent materials, the present invention distinguishes between completely transparent objects and semi-transparent objects, and uses their respective HSV values ​​to separate and extract contours. Finally, the regional growing algorithm is used to perform texture, color, and distance similarity constraints on the extracted contour information and the interior of the object. Finally, the obtained segmented fragments are fused to obtain the final object. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented area of ​​transparent objects.

[0101] The transparent object extraction device provided by the present invention is described below. The transparent object extraction device described below and the transparent object extraction method described above can be referenced to each other.

[0102] like Fig.17 As shown, the transparent object extraction device includes a preliminary segmentation module 1701, an HSV segmentation module 1702 and an extraction module 1703; The preliminary segmentation module 1701 is used to perform edge extraction on the grayscale image of the image to be segmented, and determine a first area representing the object to be segmented and a second area representing the background in the image to be segmented based on the extraction result; When transparent objects are used as detection objects, facing complex lighting conditions and changing backgrounds, due to the inconsistency of lighting, it is difficult to separate the transparent objects from the background in the detection image, making the shape of the segmented transparent objects incomplete.

[0103] To this end, the present invention introduces HSV (Hue, Saturation, Value) information into the transparent object segmentation method, in order to achieve more accurate image segmentation based on the grayscale image, so as to more accurately extract transparent objects.

[0104] To achieve image segmentation based on HSV information, it is necessary to determine a suitable HSV mask to screen the hue, saturation and brightness of the pixels, and consider the pixels within the range of the HSV mask as the segmentation results. It can be understood that the range of the HSV mask, that is, the upper and lower limits of the HSV mask, has a great influence on the image segmentation results based on HSV.

[0105] Since HSV information contains information of three channels, namely hue, saturation and brightness, when the mask range of one channel is adjusted during the process of selecting a suitable HSV mask, the ranges of the other two channels will also change, making it difficult to determine the range of the HSV mask.

[0106] Therefore, in order to achieve the transparent object extraction task based on grayscale information and HSV information, the present invention determines a suitable HSV mask in combination with the grayscale information of the object and the background in the image to be segmented.

[0107] The image to be segmented may be a color image obtained by various types of photographing equipment and containing a transparent object to be identified.

[0108] Specifically, the image to be segmented is converted into a grayscale image, such as Figure 2 As shown, an edge detection algorithm is applied to the grayscale image of the image to be segmented to realize edge extraction of the object to be segmented.

[0109] Grayscale images mainly highlight areas with large brightness changes. Since the translucent material itself will reflect, refract and transmit light, there will be a brightness difference between its brightness and the light reflected by the background itself. This difference can be used to extract contours and distinguish between objects and backgrounds.

[0110] Optionally, the edge detection algorithm uses the Canny algorithm. Specifically, a 5×5 Gaussian kernel is used to perform Gaussian filtering on the image to be segmented to remove noise; then the image gradient is calculated: ; ; ; ; In the formula, l Represents the gray value of the pixel of the image to be segmented, G x and G y Represent the horizontal and vertical gradients respectively, θ is the gradient direction, and the position where the gray value changes greatly is found by calculating the gradient as the identified edge.

[0111] On this basis, maximum suppression is performed, edge preservation is performed with double thresholds, and finally the edges are connected to obtain the rough outline of the object to be segmented, such as Figure 3 shown.

[0112] The image to be segmented is divided based on the rough contour to be segmented, and a first region representing the object to be segmented and a second region representing the background are determined.

[0113] It can be understood that the division based on the rough contour is a rough division step, so that the divided first area can roughly cover the object area. Figure 3 As an example, the edge extraction result can be determined to completely surround Figure 3 For a regular polygon with a central edge, the area inside the regular polygon is regarded as the first area representing the object, and the area outside the regular polygon is regarded as the second area representing the background.

[0114] An HSV segmentation module 1702 is used to determine an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented; Optionally, the grayscale information represents information extracted from the grayscale image, and may include a grayscale peak value and a grayscale average value, etc.

[0115] On this basis, the grayscale information of the first region represents the characteristics of the object to be segmented in the grayscale space, and the grayscale information of the second region represents the characteristics of the background in the grayscale space.

[0116] The grayscale information of the first area is compared with the grayscale information of the second information, and the HSV mask used in HSV space segmentation is determined according to the comparison result, that is, a suitable HSV mask is determined based on the characteristics of the object to be segmented and the background in the grayscale space to segment the fourth area in the HSV space.

[0117] The fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented.

[0118] In a feasible implementation, convolution is performed on the first region, a portion of the first region representing an edge of the object to be segmented is extracted as the fifth region, and a difference between the image to be segmented and the fifth region is taken as the fourth region.

[0119] On this basis, the third region is the result of segmenting the fourth region using the HSV mask, which can be understood as representing the region other than the edge of the object.

[0120] The extraction module 1703 is used to apply a region growing algorithm to the first region and the third region to obtain a target image of the image to be segmented.

[0121] The region growing algorithm is a method for image segmentation based on pixel similarity. The basic idea is to start from one or more seed points and merge adjacent pixels into the same region according to similarity criteria (such as grayscale value or color value) until there are no adjacent pixels that meet the criteria.

[0122] It can be understood that, through the above processing method, the first region is the region that characterizes the outline of the object, and the third region can be considered to be the region that roughly characterizes the interior of the object. Therefore, by applying the regional growing algorithm to the first region and the third region respectively, and merging their respective expansion results, the target area corresponding to the complete object can be obtained.

[0123] Optionally, initial seed points are sown in the first region and the third region respectively, and respective regional growth criteria are determined, and region expansion is performed based on the initial seed points. In the absence of adjacent pixels that meet the criteria, the result of regional growth is obtained, and the result is used as the final extracted target image representing the object to be segmented.

[0124] It is understandable that since the third area is the area representing the object to be segmented based on the HSV mask in the HSV space, and the HSV space highlights the color, color saturation and brightness, a single grayscale image cannot provide background information of similar colors, where color saturation and brightness will differ due to the incident angle of light. At the same time, due to the influence of transparent objects on light intensity, the brightness of different positions of the transparent objects is different, which differs from the uniform brightness and color of the background.

[0125] Therefore, it can be considered that, compared with the common contour division based on grayscale value, the third region combines the HSV information of the image to be segmented, thereby obtaining a more accurate object region segmentation result in the task of extracting transparent objects. On this basis, region growing is performed based on the third region, and a more accurate decomposition result that characterizes the contour of the object to be segmented can be obtained, and the object to be segmented can be better separated from the background.

[0126] The present invention combines the grayscale image and the HSV space to extract the unique information of the transparent object when facing light and separate it from the background, so as to obtain the third area representing the object. The region growing algorithm is applied to the first area and the third area to obtain the target image representing the image to be segmented. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented area of ​​the transparent object.

[0127] Fig.18 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.18As shown, the electronic device may include: a processor 1810, a communication interface 1820, a memory 1830 and a communication bus 1840, wherein the processor 1810, the communication interface 1820 and the memory 1830 communicate with each other through the communication bus 1840. The processor 1810 may call the logic instructions in the memory 830 to execute the transparent object extraction method, which includes: performing edge extraction on the grayscale image of the image to be segmented, and determining the first region representing the object to be segmented and the second region representing the background in the image to be segmented based on the extraction result; determining the HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segmenting the fourth region in the HSV space based on the HSV mask to obtain the third region representing the object to be segmented, wherein the fourth region represents the region of the image to be segmented after removing the edge of the image to be segmented; applying the region growing algorithm to the first region and the third region together to obtain the target image of the image to be segmented.

[0128] In addition, the logic instructions in the above-mentioned memory 1830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0129] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the transparent object extraction method provided by the above-mentioned methods, which includes: performing edge extraction on the grayscale image of the image to be segmented, and determining a first area representing the object to be segmented and a second area representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on the grayscale information of the first area and the second area, and segmenting the fourth area in the HSV space based on the HSV mask to obtain a third area representing the object to be segmented, wherein the fourth area represents the area of ​​the image to be segmented after removing the edge of the image to be segmented; and applying a region growing algorithm to the first area and the third area together to obtain a target image of the image to be segmented.

[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the transparent object extraction method provided by the above-mentioned methods, the method comprising: performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segmenting the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents the region of the image to be segmented after removing the edge of the image to be segmented; and applying a region growing algorithm to the first region and the third region together to obtain a target image of the image to be segmented.

[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting a transparent object, characterized in that: include: Performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; Determine an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; A region growing algorithm is applied to the first region and the third region together to obtain a target image of the image to be segmented.

2. The transparent object extraction method according to claim 1, characterized in that: The grayscale information includes a grayscale peak value and a grayscale average value. The step of determining the HSV mask of the image to be segmented based on the grayscale information of the first area and the second area specifically includes: Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peaks of the first area and the second area; In the case where the object to be segmented is a completely transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the first area; In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

3. The transparent object extraction method according to claim 2, characterized in that: The step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first area and the second area specifically includes: When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object; Otherwise, it is determined that the object to be segmented is a semi-transparent object.

4. The transparent object extraction method according to any one of claims 1 to 3, characterized in that: The region growing criteria of the region growing algorithm include a texture similarity criterion, an HSV value similarity criterion and a pixel point distance criterion.

5. The transparent object extraction method according to claim 4, characterized in that: The step of applying a region growing algorithm to the first region and the third region together to obtain the target image of the image to be segmented specifically includes: Convert the first region to the HSV space, perform an opening operation on the first region, and perform a convolution based on the HSV mask, and during the convolution process, spread a first preset number of seed points as initial seed points of the part according to a preset spreading method; After determining the third area, performing convolution based on the HSV mask on the third area, and during the convolution process, sowing a second preset number of seed points according to the preset sowing method as initial seed points of another part; Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

6. The transparent object extraction method according to claim 5, characterized in that: The preset spreading method is: random spreading in the first area and the third area respectively.

7. The transparent object extraction method according to claim 6, characterized in that: The step of implementing region expansion based on the initial seed point and the region growth criterion specifically includes: Based on the initial seed point of the first region, implementing region expansion according to the pixel distance criterion and the texture similarity criterion; In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion and the HSV value similarity criterion.

8. A transparent object extraction device, characterized in that: include: A preliminary segmentation module, used for performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; An HSV segmentation module, configured to determine an HSV mask of the image to be segmented based on the grayscale information of the first region and the second region, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; The extraction module is used to apply a region growing algorithm to the first region and the third region to obtain a target image of the image to be segmented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the transparent object extraction method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transparent object extraction method according to any one of claims 1 to 7 is implemented.

Images