Transparent object extraction method and apparatus

Abstract

The application provides a transparent object extraction method and device, which comprises the following steps: performing edge extraction on a gray image of a to-be-segmented image, and determining a first region representing a to-be-segmented object and a second region representing a background in the to-be-segmented image based on the extraction result; determining an HSV mask of the to-be-segmented image based on the gray information of the first region and the second region, and performing segmentation on a fourth region in HSV space based on the HSV mask to obtain a third region representing the to-be-segmented object, wherein the fourth region represents a region of the to-be-segmented image after removing the edges of the to-be-segmented image; and applying a region growing algorithm to the first region and the third region to obtain a target image of the to-be-segmented image. The application separates the unique information of the transparent object from the background by combining the gray image and the HSV space when the transparent object is exposed to light.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a method and device for extracting transparent objects. Background Art

[0002] Due to their inherent optical properties, transparent objects are difficult to distinguish from the background and other objects, making them difficult to use in situations requiring fast response characteristics, such as obstacle avoidance detection, transparent container sorting and operation, etc.

[0003] Many researchers use multispectral imaging to improve the contrast between transparent objects and the background by using light sources of different wavelengths, but this requires specialized equipment. Scholars use deep learning technology to learn the characteristics of transparent objects and improve the accuracy of transparent object extraction, but this requires a large number of samples. To this end, they use different perspectives and lighting conditions to synthesize images to enhance data diversity, but data labeling is time-consuming. Experts pre-process images to improve the separation of transparent objects from the background, but the separation values ​​need to be manually fine-tuned for different object types and cannot be updated in real time. In recent years, some scholars have used grayscale images to distinguish the brightness values ​​of transparent objects from the background under illumination, thereby separating the background. However, due to the singleness of the separation conditions and the stringent lighting requirements, this method cannot adapt to the loss of contours caused by low contrast and blurred edge information.

[0004] Therefore, how to quickly and accurately extract transparent objects remains a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a method and device for extracting transparent objects, which are used to solve the defect in the prior art that it is difficult to accurately extract transparent objects, and realize a method for extracting transparent objects quickly and accurately.

[0006] The present invention provides a transparent object extraction method, comprising:

[0007] Performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result;

[0008] determining an HSV mask of the image to be segmented based on grayscale information of the first region and the second region, and performing segmentation in a fourth region of the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented;

[0009] A region growing algorithm is applied to both the first region and the third region to obtain a target image of the image to be segmented.

[0010] According to a transparent object extraction method provided by the present invention, the grayscale information includes a grayscale peak value and a grayscale average value. The step of determining the HSV mask of the image to be segmented based on the grayscale information of the first region and the second region specifically includes:

[0011] Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on grayscale peaks of the first area and the second area;

[0012] In the case where the object to be segmented is a completely transparent object, an upper limit of the HSV mask is determined according to a grayscale peak value of the first region, and a lower limit of the HSV mask is determined according to an average grayscale value of the first region;

[0013] In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

[0014] According to a transparent object extraction method provided by the present invention, the step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first region and the second region specifically includes:

[0015] When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object;

[0016] Otherwise, it is determined that the object to be segmented is a semi-transparent object.

[0017] According to a transparent object extraction method provided by the present invention, the region growing criteria of the region growing algorithm include a texture similarity criterion, an HSV value similarity criterion and a pixel distance criterion.

[0018] According to a transparent object extraction method provided by the present invention, the step of applying a region growing algorithm to both the first region and the third region to obtain the target image of the image to be segmented specifically includes:

[0019] Converting the first region to HSV space, performing an opening operation on the first region, and performing convolution based on the HSV mask, and sowing a first preset number of seed points as initial seed points of the portion according to a preset sowing method during the convolution process;

[0020] After determining the third area, performing convolution on the third area based on the HSV mask, and sowing a second preset number of seed points as initial seed points according to the preset sowing method during the convolution process;

[0021] Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

[0022] According to a transparent object extraction method provided by the present invention, the preset spreading mode is: random spreading in the first area and the third area respectively.

[0023] According to a transparent object extraction method provided by the present invention, the step of implementing region expansion based on the region growing criterion based on the initial seed point specifically includes:

[0024] Based on the initial seed point in the first area, implementing area expansion according to the pixel point distance criterion and the texture similarity criterion;

[0025] In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion, and the HSV value similarity criterion.

[0026] The present invention also provides a transparent object extraction device, comprising:

[0027] a preliminary segmentation module, configured to perform edge extraction on a grayscale image of an image to be segmented, and determine, based on the extraction result, a first region representing an object to be segmented and a second region representing a background in the image to be segmented;

[0028] an HSV segmentation module, configured to determine an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented;

[0029] An extraction module is used to apply a region growing algorithm to the first region and the third region to obtain a target image of the image to be segmented.

[0030] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the transparent object extraction methods described above is implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the transparent object extraction methods described above.

[0032] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the transparent object extraction methods described above.

[0033] The transparent object extraction method and device provided by the present invention, by combining grayscale images and HSV space, extract the unique information of transparent objects when facing light and separate it from the background, obtain a third region representing the object, and apply the region growing algorithm to the first and third regions to obtain a target image representing the image to be segmented. It can cope with different lighting conditions and complex background information and improve the integrity of the transparent object segmentation region. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is one of the flow charts of the transparent object extraction method provided by the present invention;

[0036] Figure 2 is a grayscale image of the image to be segmented in the transparent object extraction method provided by the present invention;

[0037] Figure 3 It is a rough outline map of the image to be segmented in the transparent object extraction method provided by the present invention;

[0038] Figure 4 It is the grayscale peak image corresponding to the completely transparent object and the background in the transparent object extraction method provided by the present invention;

[0039] Figure 5 It is the grayscale peak image corresponding to the semi-transparent object and the background in the transparent object extraction method provided by the present invention;

[0040] Figure 6 is the HSV intensity histogram of the image to be segmented in the transparent object extraction method provided by the present invention;

[0041] Figure 7 is an HSV intensity histogram of the first region corresponding to the completely transparent object in the transparent object extraction method provided by the present invention;

[0042] Figure 8 is an HSV intensity histogram of the first region corresponding to the semi-transparent object in the transparent object extraction method provided by the present invention;

[0043] Figure 9is the HSV intensity histogram of the second region in the transparent object extraction method provided by the present invention;

[0044] Figure 10 This is a schematic diagram of the transparent object extraction method provided by the present invention after the first area is opened and operated;

[0045] Figure 11 Schematic diagram of the change in the number of initial seed points in the transparent object extraction method provided by the present invention;

[0046] Figure 12 Schematic diagram of the change of the initial seed point sowing method in the transparent object extraction method provided by the present invention;

[0047] Figure 13 Schematic diagram of the result of growing the contour region of the object to be segmented in the transparent object extraction method provided by the present invention;

[0048] Figure 14 Schematic diagram of the internal region growth result of the object to be segmented in the transparent object extraction method provided by the present invention;

[0049] Figure 15 This is a schematic diagram of the target area extracted by the transparent object extraction method provided by the present invention;

[0050] Figure 16 This is the second flow chart of the transparent object extraction method provided by the present invention;

[0051] Figure 17 It is a structural schematic diagram of the transparent object extraction device provided by the present invention;

[0052] Figure 18 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0054] The following combination Figures 1 to 16 The transparent object extraction method of the present invention is introduced as follows: Figure 1 Shown, including:

[0055] Step 101, performing edge extraction on a grayscale image of an image to be segmented, and determining a first region representing an object to be segmented and a second region representing a background in the image to be segmented based on the extraction result;

[0056] When transparent objects are used as detection objects, facing complex lighting conditions and changing backgrounds, due to the inconsistency of lighting, it is difficult to separate the transparent objects from the background in the detection image, resulting in incomplete shapes of the segmented transparent objects.

[0057] To this end, the present invention introduces HSV (Hue, Saturation, Value) information into the transparent object segmentation method, in order to achieve more accurate image segmentation based on the grayscale image, so as to more accurately extract transparent objects.

[0058] To achieve image segmentation based on HSV information, it is necessary to determine an appropriate HSV mask to filter the hue, saturation, and brightness of the pixels. Pixels within the HSV mask range are considered segmentation results. It is understandable that the range of the HSV mask, that is, the upper and lower limits of the HSV mask, have a significant impact on the results of HSV-based image segmentation.

[0059] Since HSV information contains information of three channels: hue, saturation, and lightness, when selecting a suitable HSV mask, when the mask range of one channel is adjusted, the ranges of the other two channels will also change, making the range of the HSV mask difficult to determine.

[0060] Therefore, in order to achieve the transparent object extraction task based on grayscale information and HSV information, the present invention determines a suitable HSV mask in combination with the grayscale information of the object and background in the image to be segmented.

[0061] The image to be segmented may be a color image containing a transparent object to be identified, acquired by various types of photographing equipment.

[0062] Specifically, the image to be segmented is converted into a grayscale image, such as Figure 2 As shown, an edge detection algorithm is applied to the grayscale image of the image to be segmented to extract the edges of the object to be segmented.

[0063] Grayscale images mainly highlight areas with large brightness changes. Since translucent materials themselves reflect, refract, and transmit light, there is a brightness difference between their brightness and the light reflected from the background itself. This difference can be used to extract contours and distinguish between objects and backgrounds.

[0064] Optionally, the Canny algorithm is used as the edge detection algorithm. Specifically, a 5×5 Gaussian kernel is used to perform Gaussian filtering on the image to be segmented to remove noise, and then the image gradient is calculated:

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] Where, l Represents the grayscale value of the pixel of the image to be segmented, G x and G y Represent the horizontal and vertical gradients respectively, θ is the gradient direction, and the position where the gray value changes greatly is found by calculating the gradient as the identified edge.

[0070] On this basis, maximum suppression is performed, edge preservation is performed with double thresholds, and finally the edges are connected to obtain the rough outline of the object to be segmented, such as Figure 3 shown.

[0071] The image to be segmented is divided based on the rough outline to be segmented, and a first region representing the object to be segmented and a second region representing the background are determined.

[0072] It can be understood that the division based on the rough contour is a rough division step, so that the first divided area can roughly cover the object area. Figure 3 As an example of the edge extraction result, it can be determined that it can completely surround Figure 3 For a regular polygon with a central edge, the area inside the regular polygon is regarded as the first area representing the object, and the area outside the regular polygon is regarded as the second area representing the background.

[0073] Step 102: Determine an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing edges of the image to be segmented.

[0074] Optionally, the grayscale information represents information extracted from the grayscale image, and may include a grayscale peak value and a grayscale average value.

[0075] On this basis, the grayscale information of the first region represents the characteristics of the object to be segmented in the grayscale space, and the grayscale information of the second region represents the characteristics of the background in the grayscale space.

[0076] The grayscale information of the first area is compared with the grayscale information of the second information, and the HSV mask used in HSV space segmentation is determined according to the comparison result. That is, a suitable HSV mask is determined based on the characteristics of the object to be segmented and the background in the grayscale space to segment the fourth area in the HSV space.

[0077] The fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented.

[0078] In a feasible implementation, convolution is performed on the first region, and a portion of the first region representing the edge of the object to be segmented is extracted as the fifth region. The difference between the image to be segmented and the fifth region is used as the fourth region.

[0079] On this basis, the third region is the result of segmenting the fourth region using the HSV mask, which can be understood as representing the region other than the edge of the object.

[0080] Step 103: Apply a region growing algorithm to both the first region and the third region to obtain a target image of the image to be segmented.

[0081] The region growing algorithm is a method for image segmentation based on pixel similarity. The basic idea is to start with one or more seed points and, based on a similarity criterion (such as grayscale or color value), merge adjacent pixels into the same region until no more adjacent pixels meet the criterion.

[0082] It can be understood that through the above processing method, the first area is the area that represents the outline of the object, and the third area can be considered as the area that roughly represents the interior of the object. Therefore, by applying the region growing algorithm to the first area and the third area respectively, and merging their respective expansion results, the target area corresponding to the complete object can be obtained.

[0083] Optionally, initial seed points are sown in the first region and the third region respectively, and respective region growth criteria are determined. Region expansion is performed based on the initial seed points. When there are no adjacent pixels that meet the criteria, the result of region growth is obtained, and the result is used as the final extracted target image representing the object to be segmented.

[0084] It is understandable that since the third area is the area representing the object to be segmented based on the HSV mask segmentation in the HSV space, and the HSV space highlights the color, color saturation and brightness, a single grayscale image cannot provide background information of similar colors, where the color saturation and brightness will differ due to the incident angle of light. At the same time, due to the influence of transparent objects on the light intensity, the brightness of different positions of the transparent objects is different, which is different from the uniform brightness and color of the background.

[0085] Therefore, it can be considered that the third region combines the HSV information of the image to be segmented, compared with the common grayscale value-based contour division, thereby obtaining a more accurate object region segmentation result in the task of extracting transparent objects. On this basis, region growing is performed based on the third region, and a more accurate decomposition result representing the contour of the object to be segmented can be obtained, which can better separate the object to be segmented from the background.

[0086] The present invention combines grayscale images and HSV space to extract unique information of transparent objects when facing light and separate it from the background, obtaining a third region representing the object. The region growing algorithm is applied to the first and third regions to obtain a target image representing the image to be segmented. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented region of transparent objects.

[0087] In the transparent object extraction method of the present invention, the grayscale information includes a grayscale peak value and a grayscale average value. The step of determining the HSV mask of the image to be segmented based on the grayscale information of the first region and the second region specifically includes:

[0088] Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on grayscale peaks of the first area and the second area;

[0089] Furthermore, the category of the object to be segmented is determined to be a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first region and the second region.

[0090] Specifically, there is a difference in the grayscale peaks between the background and the object, which is caused by the reflection intensity of the material to light. Usually, e.g. Figure 4 and Figure 5 As shown in FIG, for a completely transparent object, the grayscale peak value of the object is greater than the grayscale peak value of the background; for a semi-transparent object, the grayscale peak value of the object is less than the grayscale peak value of the background.

[0091] Based on this, it can be determined whether the object to be segmented is a completely transparent object or a semi-transparent object through the grayscale peak values ​​of the first area and the second area.

[0092] On this basis, the HSV mask corresponding to the object to be segmented is determined according to the category of the object to be segmented. The HSV intensity histograms of the image to be segmented, the first area corresponding to the completely transparent object, the first area corresponding to the semi-transparent object, and the second area corresponding to the background are respectively as follows: Figure 6 、 Figure 7 、 Figure 8 and Figure 9 shown.

[0093] In the case where the object to be segmented is a completely transparent object, an upper limit of the HSV mask is determined according to a grayscale peak value of the first region, and a lower limit of the HSV mask is determined according to an average grayscale value of the first region;

[0094] When the object to be segmented is a completely transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first region.

[0095] Specifically, in the grayscale image of the object to be segmented, the coordinates of all pixel points corresponding to the grayscale peak values ​​of the first region are determined and recorded as grayscale coordinates.

[0096] The grayscale coordinates are mapped back to the original image to be segmented, and the pixels corresponding to the grayscale coordinates in the image to be segmented are obtained and organized into a first pixel set.

[0097] Obtain an intensity distribution histogram corresponding to H, S, and V of each pixel in the first pixel set.

[0098] It can be understood that since the pixel points in the first pixel set are all pixel points corresponding to the grayscale peak value of the first region, the H values, S values ​​and V values ​​of the pixel points in the first pixel set are all concentratedly distributed. The median, mode or mean of their H values, S values ​​and V values ​​can be taken as the HSV value of the first pixel set, and the determined HSV value can be used as the upper limit of the HSV mask.

[0099] The method of determining the lower limit of the HSV mask based on the grayscale average value of the first area is similar. The grayscale average value of the first area is calculated, and the coordinates of the grayscale average value of the first area in the grayscale image are determined. After mapping it back to the image to be segmented, the pixels corresponding to the coordinates are extracted and organized into a second pixel set.

[0100] Obtain an intensity distribution histogram corresponding to H, S, and V for each pixel in the second pixel set, and determine the lower limit of the HSV mask.

[0101] In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

[0102] When the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first region, the grayscale average value of the second region is calculated, and the lower limit of the HSV mask is determined according to the grayscale average value of the second region.

[0103] The upper and lower limits are determined in the same manner as the upper and lower limits of the HSV mask for a completely transparent object, and therefore will not be described in detail.

[0104] It can be understood that by determining the upper and lower limits of the HSV mask, that is, determining the screening range of the HSV mask, the object and the background can be separated by screening in the HSV space corresponding to the first area, thereby obtaining the third area.

[0105] The present invention provides additional channel information to compensate for the constraint of a single grayscale value by matching the HSV value of the pixel corresponding to the maximum grayscale peak with the HSV value of the original image, thereby greatly improving the segmentation effect.

[0106] In the transparent object extraction method of the present invention, the step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first region and the second region specifically includes:

[0107] When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object;

[0108] Otherwise, it is determined that the object to be segmented is a semi-transparent object.

[0109] Determine the grayscale peak of the first region P i , used to represent the grayscale peak of the object to be segmented.

[0110] Determine the grayscale peak of the second area P b , used to characterize the grayscale peak of the background.

[0111] exist P i > P b In the case of , the object to be segmented is considered to be a completely transparent object; otherwise, the object to be segmented is considered to be a semi-transparent object.

[0112] In the transparent object extraction method of the present invention, the region growing criteria of the region growing algorithm include a texture similarity criterion, an HSV value similarity criterion and a pixel distance criterion.

[0113] In the region growing algorithm, it is necessary to predetermine the region growing criterion to be followed by the region expansion. Since the method of the present invention introduces the characteristics of the HSV space, the corresponding defined region growing criterion also includes the HSV value similarity criterion.

[0114] Alternatively, the HSV value similarity between pixels can be expressed by the following formula:

[0115] ;

[0116] Where, It is a weight that adapts to the different brightness and luminance caused by reflection and refraction under similar colors.

[0117] Will D HSV Two pixels whose HSV value is less than the preset HSV threshold are considered to be similar and can be merged.

[0118] In addition, the defined region growing criteria also include texture similarity criteria, which specifically calculates the similarity of local textures through local binary patterns (LBP) to adapt to complex environmental background information. When the LBP value of the surrounding points and the seed point is less than the set threshold, expansion can be performed.

[0119] In addition, the defined region growing criterion also includes the pixel distance criterion to ensure the connectivity of the internal region of the object, which can be expressed by the following formula:

[0120] ;

[0121] Where, P is the pixel coordinate.

[0122] It should be noted that, among the three criteria defined in this embodiment, one or more criteria may be selected for use according to needs at different locations of area expansion.

[0123] In the transparent object extraction method of the present invention, the step of applying a region growing algorithm to both the first region and the third region to obtain the target image of the image to be segmented specifically includes:

[0124] After performing an opening operation on the contour of the third region, performing convolution based on the HSV mask, and during the convolution process, sowing a first preset number of seed points as initial seed points of the portion according to a preset sowing method;

[0125] Converting the first region to HSV space, performing an opening operation on the first region, and performing convolution based on the HSV mask, and sowing a first preset number of seed points as initial seed points of the portion according to a preset sowing method during the convolution process;

[0126] In order to avoid omission of the segmented object area during the region growing process, a certain number of initial seed points are sown in the third region representing the interior of the object and the first region representing the outline of the object, and the region growing operation is completed in these two regions at the same time.

[0127] Specifically, the spreading process of the first region is performed after the first region is determined. Since the first region is obtained based on the grayscale image of the image to be segmented, the first region needs to be converted into the HSV space first.

[0128] On this basis, the first region in the HSV space is opened to avoid omissions in the process of extracting the edge of the object to be segmented. The first region after the opening operation is as follows: Figure 10 shown.

[0129] During the opening operation, a 5×5 mask is used to perform a mask convolution operation as the first mask convolution, so that a more accurate area for characterizing the edge of the object can be further separated in the first area according to the determined HSV mask.

[0130] During the convolution process, a first preset number of seed points are spread as a part of the initial seed points according to a preset spreading method. Thus, the spreading of the initial seed points in the first area representing the contour of the object is completed.

[0131] After determining the third area, performing convolution on the third area based on the HSV mask, and during the convolution process, sowing a second preset number of seed points as initial seed points for another part according to the preset sowing method;

[0132] Specifically, the sowing process for the third area is performed after the third area is determined.

[0133] It should be noted that since the third region is the result of segmenting the fourth region, the fourth region is the part of the image to be segmented without the fifth region representing the edge of the object, and in this embodiment, a more accurate object edge is obtained based on HSV information extraction after the first convolution is performed on the first region. Therefore, the result of the mask convolution of the first region is directly used as the fifth region, and the remaining part of the image to be segmented after removing the fifth region is used as the fourth region, so that the third region is obtained based on this fourth region.

[0134] After determining the third area, a 5×5 mask is used to perform a convolution operation on the third area as the second mask convolution. During the second convolution, a second preset number of seed points are sown in a preset sowing method as the initial seed points of the other part.

[0135] At this point, the initial seed point sowing of the third area representing the interior of the object is completed.

[0136] The preset sowing method predetermines the sowing position of the initial seed point. For example, it can be completely random sowing or random sowing in each predetermined area, so that the initial seeds are more evenly distributed in the entire area.

[0137] The first preset number and the second preset number, that is, the number of seed points sown twice, can be adjusted according to the specific experimental process.

[0138] In this embodiment, the exploration process of the number of initial seed points (n) to be sown is as follows: Figure 11 As shown in the figure, when n is small, the seed points can only grow in the area around them and cannot cover the entire object. When n reaches 15, the seed points are clustered into a single area (the bottom of the bowl). When n reaches 20, the shape and outline of the bowl are clearly displayed. However, the contact area between the seeds and the background increases, resulting in incorrect object segmentation (n = 25). Therefore, the initial number of seed points for both seedings was set to 20.

[0139] Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

[0140] After the initial seed point is determined, the region can be expanded based on the predefined region growing criterion based on the initial seed point.

[0141] Optionally, starting from the initial seed point, check whether its 8-neighborhood satisfies the region growing criterion. If so, add the pixel to the current region and continue to expand to other unvisited pixels.

[0142] After the expansion stops, that is, when there are no new pixels that can be expanded, an expanded image representing the object based on the third region and an expanded image representing the object contour based on the first region are obtained. The two expanded images are merged as the target image of the image to be segmented.

[0143] In the transparent object extraction method of the present invention, the preset spreading mode is: random spreading in the first area and the third area respectively.

[0144] In this embodiment, the exploration process of the preset spreading mode is as follows: Figure 12 As shown, the first area representing the outline of the object and the third area representing the interior of the object are spread respectively. On this basis, the effects of different spreading methods such as random spreading and uniform spreading on the transparent object extraction effect in the outline and interior of the object are further explored.

[0145] Specifically, when using uniform seeding for edge processing, the edges of transparent objects overlap with the background due to inconsistent light reflection. Random seeding, however, concentrates the random seeding in a specific area to produce segmentation. When processing the interior of an object, there's little difference between the two distributions. Uniform seeding requires consistent light distribution, but the reflective and transmissive properties of materials don't satisfy this requirement.

[0146] Therefore, the final choice is to randomly distribute the initial seeds around the outline and inside of the object.

[0147] In other feasible implementations, different sowing methods may be configured for the initial seeds in the first area and the third area of ​​the object according to actual exploration conditions.

[0148] In the transparent object extraction method of the present invention, the step of implementing region expansion based on the region growing criterion based on the initial seed point specifically includes:

[0149] Based on the initial seed point of the first region, implementing region expansion according to the pixel point distance criterion and the texture similarity criterion;

[0150] In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion, and the HSV value similarity criterion.

[0151] It can be understood that since the outline of the object and the interior of the object are distinguished and sown separately during the sowing process of the initial seed points, different region growing criteria can be further applied to the outline of the object and the interior of the object to achieve differentiated region expansion during region expansion.

[0152] Specifically, since the interior of an object and the edge of its contour are of different sizes, the growth constraints are relaxed during contour processing, and the pixel distance criterion and texture similarity criterion are adopted as the main constraints. That is to say, when processing contours, pixels that meet the pixel distance criterion and texture similarity criterion can be merged and expanded.

[0153] When dealing with the interior of an object, the three defined criteria must be met simultaneously before they can be merged and expanded.

[0154] According to the above processing method, after the region expansion is completed, the target segment corresponding to the outline of the object and the target segment corresponding to the interior of the object are obtained respectively, such as Figure 13 and Figure 14 As shown in , the target region can be obtained by merging the two target segments, as shown in Figure 15 It should be noted that in order to more clearly show the mentioned regions in the drawings, black and white drawings are used, and the region growing process should be completed in the original image to be segmented, that is, it should be completed in the image containing HSV information.

[0155] On this basis, a complete process of extracting the target area of ​​the object to be segmented in the image to be segmented is as follows: Figure 16 shown.

[0156] The present invention combines grayscale images and HSV space to extract the unique information of transparent objects when facing light and separate them from the background. In view of the different light transmission intensities of translucent materials, the present invention distinguishes between completely transparent objects and semi-transparent objects, and uses their respective HSV values ​​to separate and extract contours. Finally, the extracted contour information and the interior of the object are constrained by texture, color, and distance similarity using the region growing algorithm. Finally, the obtained segmented fragments are fused to obtain the final object. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented area of ​​transparent objects.

[0157] The transparent object extraction device provided by the present invention is described below. The transparent object extraction device described below and the transparent object extraction method described above can be referenced to each other.

[0158] like Figure 17 As shown, the transparent object extraction device includes a preliminary segmentation module 1701, an HSV segmentation module 1702 and an extraction module 1703;

[0159] A preliminary segmentation module 1701 is configured to perform edge extraction on a grayscale image of an image to be segmented, and determine a first region representing an object to be segmented and a second region representing a background in the image to be segmented based on the extraction result;

[0160] When transparent objects are used as detection objects, facing complex lighting conditions and changing backgrounds, due to the inconsistency of lighting, it is difficult to separate the transparent objects from the background in the detection image, resulting in incomplete shapes of the segmented transparent objects.

[0161] To this end, the present invention introduces HSV (Hue, Saturation, Value) information into the transparent object segmentation method, in order to achieve more accurate image segmentation based on the grayscale image, so as to more accurately extract transparent objects.

[0162] To achieve image segmentation based on HSV information, it is necessary to determine an appropriate HSV mask to filter the hue, saturation, and brightness of the pixels. Pixels within the HSV mask range are considered segmentation results. It is understandable that the range of the HSV mask, that is, the upper and lower limits of the HSV mask, have a significant impact on the results of HSV-based image segmentation.

[0163] Since HSV information contains information of three channels: hue, saturation, and lightness, when selecting a suitable HSV mask, when the mask range of one channel is adjusted, the ranges of the other two channels will also change, making the range of the HSV mask difficult to determine.

[0164] Therefore, in order to achieve the transparent object extraction task based on grayscale information and HSV information, the present invention determines a suitable HSV mask in combination with the grayscale information of the object and background in the image to be segmented.

[0165] The image to be segmented may be a color image containing a transparent object to be identified, acquired by various types of photographing equipment.

[0166] Specifically, the image to be segmented is converted into a grayscale image, such as Figure 2 As shown, an edge detection algorithm is applied to the grayscale image of the image to be segmented to extract the edges of the object to be segmented.

[0167] Grayscale images mainly highlight areas with large brightness changes. Since translucent materials themselves reflect, refract, and transmit light, there is a brightness difference between their brightness and the light reflected from the background itself. This difference can be used to extract contours and distinguish between objects and backgrounds.

[0168] Optionally, the Canny algorithm is used as the edge detection algorithm. Specifically, a 5×5 Gaussian kernel is used to perform Gaussian filtering on the image to be segmented to remove noise; then the image gradient is calculated:

[0169] ;

[0170] ;

[0171] ;

[0172] ;

[0173] Where, l Represents the grayscale value of the pixel of the image to be segmented, G x and G y Represent the horizontal and vertical gradients respectively, θ is the gradient direction, and the position where the gray value changes greatly is found by calculating the gradient as the identified edge.

[0174] On this basis, maximum suppression is performed, edge preservation is performed with double thresholds, and finally the edges are connected to obtain the rough outline of the object to be segmented, such as Figure 3 shown.

[0175] The image to be segmented is divided based on the rough outline to be segmented, and a first region representing the object to be segmented and a second region representing the background are determined.

[0176] It can be understood that the division based on the rough contour is a rough division step, so that the first divided area can roughly cover the object area. Figure 3As an example of the edge extraction result, it can be determined that it can completely surround Figure 3 For a regular polygon with a central edge, the area inside the regular polygon is regarded as the first area representing the object, and the area outside the regular polygon is regarded as the second area representing the background.

[0177] An HSV segmentation module 1702 is configured to determine an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented;

[0178] Optionally, the grayscale information represents information extracted from the grayscale image, and may include a grayscale peak value and a grayscale average value.

[0179] On this basis, the grayscale information of the first region represents the characteristics of the object to be segmented in the grayscale space, and the grayscale information of the second region represents the characteristics of the background in the grayscale space.

[0180] The grayscale information of the first area is compared with the grayscale information of the second information, and the HSV mask used in HSV space segmentation is determined according to the comparison result. That is, a suitable HSV mask is determined based on the characteristics of the object to be segmented and the background in the grayscale space to segment the fourth area in the HSV space.

[0181] The fourth region represents a region of the image to be segmented after removing the edge of the image to be segmented.

[0182] In a feasible implementation, convolution is performed on the first region, and a portion of the first region representing the edge of the object to be segmented is extracted as the fifth region. The difference between the image to be segmented and the fifth region is used as the fourth region.

[0183] On this basis, the third region is the result of segmenting the fourth region using the HSV mask, which can be understood as representing the region other than the edge of the object.

[0184] The extraction module 1703 is configured to apply a region growing algorithm to both the first region and the third region to obtain a target image of the image to be segmented.

[0185] The region growing algorithm is a method for image segmentation based on pixel similarity. The basic idea is to start with one or more seed points and, based on a similarity criterion (such as grayscale or color value), merge adjacent pixels into the same region until no more adjacent pixels meet the criterion.

[0186] It can be understood that through the above processing method, the first area is the area that represents the outline of the object, and the third area can be considered as the area that roughly represents the interior of the object. Therefore, by applying the region growing algorithm to the first area and the third area respectively, and merging their respective expansion results, the target area corresponding to the complete object can be obtained.

[0187] Optionally, initial seed points are sown in the first region and the third region respectively, and respective region growth criteria are determined. Region expansion is performed based on the initial seed points. When there are no adjacent pixels that meet the criteria, the result of region growth is obtained, and the result is used as the final extracted target image representing the object to be segmented.

[0188] It is understandable that since the third area is the area representing the object to be segmented based on the HSV mask segmentation in the HSV space, and the HSV space highlights the color, color saturation and brightness, a single grayscale image cannot provide background information of similar colors, where the color saturation and brightness will differ due to the incident angle of light. At the same time, due to the influence of transparent objects on the light intensity, the brightness of different positions of the transparent objects is different, which is different from the uniform brightness and color of the background.

[0189] Therefore, it can be considered that the third region combines the HSV information of the image to be segmented, compared with the common grayscale value-based contour division, thereby obtaining a more accurate object region segmentation result in the task of extracting transparent objects. On this basis, region growing is performed based on the third region, and a more accurate decomposition result representing the contour of the object to be segmented can be obtained, which can better separate the object to be segmented from the background.

[0190] The present invention combines grayscale images and HSV space to extract unique information of transparent objects when facing light and separate it from the background, obtaining a third region representing the object. The region growing algorithm is applied to the first and third regions to obtain a target image representing the image to be segmented. This method can cope with different lighting conditions and complex background information and improve the integrity of the segmented region of transparent objects.

[0191] Figure 18 An example of a physical structure diagram of an electronic device is shown below. Figure 18As shown, the electronic device may include: a processor 1810, a communication interface 1820, a memory 1830, and a communication bus 1840, wherein the processor 1810, the communication interface 1820, and the memory 1830 communicate with each other via the communication bus 1840. The processor 1810 may call logic instructions in the memory 830 to execute a transparent object extraction method, which includes: performing edge extraction on a grayscale image of an image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segmenting a fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents the region of the image to be segmented after removing the edges of the image to be segmented; and applying a region growing algorithm to the first and third regions to obtain a target image of the image to be segmented.

[0192] Furthermore, the logic instructions in the aforementioned memory 1830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0193] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the transparent object extraction method provided by the above methods, which includes: performing edge extraction on the grayscale image of the image to be segmented, and determining a first area representing the object to be segmented and a second area representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on the grayscale information of the first area and the second area, and segmenting the fourth area in the HSV space based on the HSV mask to obtain a third area representing the object to be segmented, wherein the fourth area represents the area of ​​the image to be segmented after removing the edge of the image to be segmented; and applying a region growing algorithm to the first area and the third area to obtain the target image of the image to be segmented.

[0194] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the transparent object extraction method provided by the above-mentioned methods, the method comprising: performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segmenting the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents the region of the image to be segmented after removing the edges of the image to be segmented; and applying a region growing algorithm to the first and third regions together to obtain a target image of the image to be segmented.

[0195] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0196] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for extracting transparent objects, characterized in that: include: Performing edge extraction on the grayscale image of the image to be segmented, and determining a first region representing the object to be segmented and a second region representing the background in the image to be segmented based on the extraction result; determining an HSV mask of the image to be segmented based on grayscale information of the first and second regions, and segmenting the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; Applying a region growing algorithm to both the first region and the third region to obtain a target image of the image to be segmented; The grayscale information includes a grayscale peak value and a grayscale average value. The step of determining the HSV mask of the image to be segmented based on the grayscale information of the first region and the second region specifically includes: Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on grayscale peaks of the first area and the second area; In the case where the object to be segmented is a completely transparent object, an upper limit of the HSV mask is determined according to a grayscale peak value of the first region, and a lower limit of the HSV mask is determined according to an average grayscale value of the first region; In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

2. The transparent object extraction method according to claim 1, characterized in that: The step of determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on the grayscale peak values ​​of the first area and the second area specifically includes: When the grayscale peak value of the first area is greater than the grayscale peak value of the second area, determining that the object to be segmented is a completely transparent object; Otherwise, it is determined that the object to be segmented is a semi-transparent object.

3. The transparent object extraction method according to claim 1 or 2, characterized in that: The region growing criteria of the region growing algorithm include a texture similarity criterion, an HSV value similarity criterion and a pixel distance criterion.

4. The transparent object extraction method according to claim 3, characterized in that: The step of applying a region growing algorithm to both the first region and the third region to obtain the target image of the image to be segmented specifically includes: Converting the first region to HSV space, performing an opening operation on the first region, and performing convolution based on the HSV mask, and sowing a first preset number of seed points as initial seed points of the portion according to a preset sowing method during the convolution process; After determining the third area, performing convolution on the third area based on the HSV mask, and during the convolution process, sowing a second preset number of seed points as initial seed points for another part according to the preset sowing method; Based on the initial seed point, region expansion is achieved based on the region growing criterion, and after the expansion is completed, the target image of the image to be segmented is obtained.

5. The transparent object extraction method according to claim 4, characterized in that: The preset spreading method is: random spreading in the first area and the third area respectively.

6. The transparent object extraction method according to claim 5, characterized in that: The step of implementing region expansion based on the initial seed point and the region growing criterion specifically includes: Based on the initial seed point of the first region, implementing region expansion according to the pixel point distance criterion and the texture similarity criterion; In the case where the initial seed point of the third region is used as a basis, region expansion is achieved according to the pixel point distance criterion, the texture similarity criterion, and the HSV value similarity criterion.

7. A transparent object extraction device, characterized in that: include: a preliminary segmentation module, configured to perform edge extraction on a grayscale image of an image to be segmented, and determine, based on the extraction result, a first region representing an object to be segmented and a second region representing a background in the image to be segmented; an HSV segmentation module, configured to determine an HSV mask of the image to be segmented based on the grayscale information of the first and second regions, and segment the fourth region in the HSV space based on the HSV mask to obtain a third region representing the object to be segmented, wherein the fourth region represents a region of the image to be segmented after removing an edge of the image to be segmented; an extraction module, configured to apply a region growing algorithm to both the first region and the third region to obtain a target image of the image to be segmented; The grayscale information includes grayscale peak value and grayscale average value, and the HSV segmentation module is specifically used to: Determining whether the object to be segmented is a completely transparent object or a semi-transparent object based on grayscale peaks of the first area and the second area; In the case where the object to be segmented is a completely transparent object, an upper limit of the HSV mask is determined according to a grayscale peak value of the first region, and a lower limit of the HSV mask is determined according to an average grayscale value of the first region; In the case that the object to be segmented is a semi-transparent object, the upper limit of the HSV mask is determined according to the grayscale peak value of the first area, and the lower limit of the HSV mask is determined according to the grayscale average value of the second area.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the transparent object extraction method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transparent object extraction method according to any one of claims 1 to 6 is implemented.

Images