Image segmentation method and device, electronic equipment, storage medium and chip

Abstract

The invention provides an image segmentation method and device, electronic equipment, a storage medium and a chip, and the method comprises the steps: determining an initial clustering center corresponding to each superpixel based on the image size of a to-be-segmented image and a preset number of superpixels; the number of the preset superpixels is at least two; determining a pixel point with the minimum gradient value in a preset first neighborhood of each initial clustering center as a first clustering center corresponding to each superpixel; the first clustering center refers to a clustering center after the initial clustering center of each superpixel is updated; determining a second clustering center corresponding to each pixel point in the to-be-segmented image based on the RGB color distance and the spatial distance between each pixel point in the to-be-segmented image and each first clustering center; and obtaining a superpixel segmentation result of the to-be-segmented image based on the second clustering center corresponding to each pixel point in the to-be-segmented image.

Description

Technical Field

[0001] The present disclosure relates to the field of image segmentation, and in particular to an image segmentation method, device, electronic device, storage medium and chip. Background Art

[0002] In the field of computer vision, image segmentation refers to the process of subdividing a digital image into multiple image sub-regions, namely superpixels. A superpixel is a small area composed of a series of adjacent pixels with similar characteristics such as color, brightness, and texture. Superpixels retain effective information for further image segmentation without destroying the boundary information of objects in the image. However, the image segmentation method in the related art requires setting a large number of parameters and has high computational complexity. Summary of the invention

[0003] The present disclosure provides an image segmentation method, device, electronic device, storage medium and chip to solve the problems in the related art.

[0004] A first aspect of the present disclosure provides an image segmentation method, the method comprising:

[0005] Determine an initial cluster center corresponding to each superpixel based on the image size of the image to be segmented and a preset number of superpixels; the preset number of superpixels is at least two;

[0006] Determine that within a preset first neighborhood of each of the initial cluster centers, a pixel point with the smallest gradient value is a first cluster center corresponding to each of the superpixels; the first cluster center refers to a cluster center after the initial cluster center of each superpixel is updated;

[0007] Determine a second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and the spatial distance between each pixel in the image to be segmented and each of the first cluster centers;

[0008] Based on the second cluster center corresponding to each pixel point in the image to be segmented, a superpixel segmentation result of the image to be segmented is obtained.

[0009] In some embodiments, determining the second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and the spatial distance between each pixel in the image to be segmented and each of the first cluster centers includes:

[0010] Based on each of the first cluster centers and a preset second neighborhood size, determining a preset second neighborhood corresponding to each of the first cluster centers; the size of the preset second neighborhood is larger than the size of the preset superpixel;

[0011] Based on the preset second neighborhood corresponding to each of the first cluster centers, determine the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood;

[0012] Based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, the second cluster center corresponding to each pixel point in the image to be segmented is determined.

[0013] In some embodiments, determining the second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and spatial distance between each pixel in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood includes:

[0014] Determine the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood;

[0015] Based on the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood, the first clustering center with the smallest clustering distance to each pixel point in the image to be segmented is determined as the second clustering center corresponding to each pixel point in the image to be segmented.

[0016] In some embodiments, determining the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the LAB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood includes:

[0017] Determine the RGB color distance weight and the spatial distance weight based on the maximum spatial distance within the preset class and the maximum color distance within the preset class;

[0018] Based on the RGB color distance weight, the spatial distance weight, and the RGB color distance and spatial distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood, the clustering distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood is determined.

[0019] In some embodiments, before obtaining the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented, the method further includes:

[0020] Using the second cluster center to update the initial cluster center, iterate, and obtain the second cluster center corresponding to each pixel in the image to be segmented after each iteration;

[0021] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met.

[0022] In some embodiments, the determining of the second cluster center corresponding to each pixel in the image to be segmented after the iteration termination condition is met includes:

[0023] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the number of iterations reaches a preset number of iteration terminations.

[0024] In some embodiments, the method further comprises:

[0025] Performing connected domain enhancement processing on the superpixel segmentation result of the image to be segmented, to obtain the superpixel segmentation result of the image to be segmented after the connected domain enhancement.

[0026] A second aspect of the present disclosure provides an image segmentation device, the device comprising:

[0027] An initial cluster center determination unit, configured to determine an initial cluster center corresponding to each superpixel based on the image size of the image to be segmented and a preset number of superpixels; the preset number of superpixels is at least two;

[0028] A first cluster center determination unit, used to determine a pixel point with the smallest gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels; the first cluster center refers to a cluster center after the initial cluster center of each superpixel is updated;

[0029] A second cluster center determination unit, configured to determine a second cluster center corresponding to each pixel point in the image to be segmented based on an RGB color distance and a spatial distance between each pixel point in the image to be segmented and each of the first cluster centers;

[0030] The superpixel segmentation result acquisition unit is used to acquire the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented.

[0031] A third aspect of the present disclosure provides an electronic device, including:

[0032] at least one processor;

[0033] and a memory communicatively coupled to the at least one processor;

[0034] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the method described in the embodiment of the first aspect of the present disclosure.

[0035] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0036] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect embodiment of the present disclosure.

[0037] In summary, the present disclosure proposes an image segmentation method, device, electronic device, storage medium and chip. The method includes determining the initial cluster center corresponding to each superpixel based on the image size of the image to be segmented and the preset number of superpixels; the preset number of superpixels is at least two; determining the pixel point with the smallest gradient value in the preset first neighborhood of each initial cluster center as the first cluster center corresponding to each superpixel; the first cluster center refers to the cluster center after the initial cluster center of each superpixel is updated; based on the RGB color distance and spatial distance between each pixel point in the image to be segmented and each of the first cluster centers, determining the second cluster center corresponding to each pixel point in the image to be segmented; based on the second cluster center corresponding to each pixel point in the image to be segmented, obtaining the superpixel segmentation result of the image to be segmented.

[0038] Compared with the Simple Linear Iterative Clustering (SLIC) algorithm used in the related art, the solution provided by the present disclosure uses RGB color distance instead of LAB color distance to determine the cluster center of each pixel in the pixels to be segmented. Since the image to be processed is an RGB image, it is not necessary to convert the image to be processed into a LAB space image, which greatly reduces the amount of calculation and saves hardware costs. At the same time, compared with the image segmentation method in the related art, the solution provided by the present disclosure only needs to set the parameters of the number of superpixels to obtain better image segmentation results, further reducing the consumption of hardware resources.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0041] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0042] Figure 2 A flowchart of an image segmentation method provided by an embodiment of the present disclosure;

[0043] Figure 3 A schematic diagram of a flow chart of a method for determining a second cluster center provided in an embodiment of the present disclosure;

[0044] Figure 4 A flowchart of an image segmentation method provided for the first application example of the present disclosure;

[0045] Figure 5 A schematic diagram of an image segmentation architecture provided for a second application example of the present disclosure;

[0046] Figure 6 A flowchart of an image segmentation method provided for the second application example of the present disclosure;

[0047] Figure 7 A schematic diagram of the structure of an image segmentation device provided in an embodiment of the present disclosure;

[0048] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0049] Fig. 9 A schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0051] The result of image segmentation is a collection of sub-regions on the image, or a collection of contours extracted from the image. These sub-regions cover the entire image, and each pixel in the sub-region is similar in terms of some characteristic metric or calculated characteristic, such as color, brightness, texture, etc. Adjacent regions are very different in terms of some characteristic metric.

[0052] Superpixels in image segmentation refer to irregular pixel blocks with certain meanings composed of adjacent pixels with similar texture, color, brightness and other characteristics. It uses the similarity of features between pixels to group pixels and replaces a large number of pixels with a small number of superpixels to express image features, which greatly reduces the complexity of image segmentation. Therefore, it is usually used as a preprocessing step of image segmentation algorithms. Superpixels mostly retain effective information for further image segmentation and do not destroy the boundary information of objects in the image.

[0053] The following two schemes are used as examples to illustrate the superpixel extraction method in the related art.

[0054] Scheme A: Scheme A discloses a superpixel extraction method, which includes step 1, determining the brightness value similarity, red-green value similarity, yellow-blue value similarity, and coordinate Euclidean distance between the pixel point and the central pixel point of the superpixel block; then determining the distance ratio function according to the brightness value similarity, red-green value similarity, yellow-blue value similarity, and coordinate Euclidean distance; calculating the similarity of two distances between the central pixel point of the superpixel block and each pixel point of the superpixel block by the distance ratio function, the similarity of the two distances is the similarity of the Lab distance between the central pixel point of each superpixel block and each pixel point of the superpixel block and the similarity of the coordinate distance between the central pixel point of each superpixel block and each pixel point of the superpixel block; then the similarity of the two distances between each pixel of the superpixel block and the central point is composed of d1, d2, d3...dn respectively, where n represents the superpixel The total number of pixels in the pixel block except the central pixel; Step 2, determine the weight m = (max(D) + min(D)) / 2; Step 3, divide the pixels in the super pixel block into two parts according to the weight m, and the distance similarity set of the two distance similarities between the pixel point and the center point greater than the weight m is recorded as m1, and the distance similarity set less than the weight m is recorded as m2; Step 4, calculate the mean P1 and P2 of m1 and m2 respectively; Step 5, calculate the new weight mm = (P1+P2) / 2; Step 6, replace the original weight m with the new weight mm, repeat steps 3, 4, 5 and 6 until the difference between the two weights d = m-mm, which is less than the allowed value allow, then the iteration ends, and the new weight mm at the end of the iteration is used as the weight of the pixel block; Step 7, calculate the weights of all super pixel blocks, and take their average as the weight of the image. According to the similarity of the two distances, image Lab distance and coordinate distance, the weights are adaptively determined using an iterative method to determine the weights of the superpixel distance formula, thereby reducing the inaccuracy of superpixel extraction and ultimately improving the effect of image superpixel extraction.

[0055] Solution B: Solution B provides a method for representing superpixels and measuring the similarity between superpixels, which includes two stages. In the first stage, superpixel construction is performed: for a given input image, superpixels are obtained to establish a superpixel pyramid, extract the largest inscribed square, solve the significant square, and establish a superpixel descriptor; in the second stage, superpixel similarity calculation based on the superpixel model is performed: on the basis of obtaining the superpixel model descriptor, the similarity between two superpixels and is measured. This solution can better match the characteristics of superpixels, accurately capture the detailed information of superpixels, and reasonably handle the diversity of superpixel separation and merging.

[0056] However, the above two solutions have the following defects:

[0057] Both Scheme A and Scheme B require a large number of parameters. For example, Scheme A requires a large number of parameters such as the brightness similarity, red-green similarity, yellow-blue similarity, and coordinate Euclidean distance between the pixel and the central pixel of the superpixel block. Scheme B requires parameters such as superpixel pyramid, maximum inscribed square, solution of significant square, superpixel descriptor, etc., and has high computational complexity.

[0058] In order to solve at least one of the above defects, the present disclosure proposes an image segmentation method.

[0059] Before introducing the detailed solution of the present disclosure, the scenario to which the solution of the present disclosure is applied is described first. Figure 1 FIG. 1 is an application scenario diagram of an image segmentation method in an embodiment. Figure 1 As shown, the application scenario includes an electronic device 101, in which a camera module and several applications can be installed. The application can initiate an image acquisition instruction to acquire an image, and the camera module acquires an image 102. Among them, the camera module can include a front camera module and / or a rear camera module. Finally, the image 102 is sent to the target application. Among them, the electronic device 101 can be a smart phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0060] In some optional embodiments, the electronic device 101 may also be a vehicle-mounted device or an Internet of Vehicles device, such as a smart car, etc. In the present disclosure, only a smart phone is taken as an example, but it does not limit the scope of the present disclosure.

[0061] A camera may be installed on the electronic device 101, and an image 102 may be acquired through the installed camera. The camera may be divided into laser cameras, visible light cameras, and other types according to the different acquired images 102. The laser camera may acquire the image 102 formed by irradiating the laser onto the object, and the visible light camera may acquire the image formed by irradiating the visible light onto the object. Several cameras may be installed on the electronic device 101, and the installation positions are not limited. For example, a camera may be installed on the front panel of the electronic device 101, and two cameras may be installed on the back panel. The camera may also be installed inside the electronic device 101 in an embedded manner, and then the camera may be opened by rotating or sliding. Specifically, a front camera and a rear camera may be installed on the electronic device 101. The front camera and the rear camera may acquire the image 102 from different perspectives. Generally, the front camera may acquire the image 102 from the front perspective of the electronic device, and the rear camera may acquire the image 102 from the back perspective of the electronic device.

[0062] It should be understood that in the present disclosure, the front camera or the rear camera is only used as an example to distinguish the shooting angles of different cameras, rather than to limit the functions of multiple cameras. The multiple cameras in the present disclosure can be rear cameras or front cameras at the same time, which is not limited in the present disclosure.

[0063] Several applications can be installed in the electronic device 101. An application refers to software written in an electronic device for a certain application purpose. The electronic device 101 can provide services to users through the application. When the application needs to capture the image 102, an image capture instruction will be initiated. The electronic device 101 will call the camera module according to the image capture instruction to capture the image 102. The image capture instruction refers to an instruction used to trigger the image capture operation.

[0064] The electronic device 101 is also provided with a processor, and the image segmentation module in the processor can reconstruct the image 102 captured by the camera module, for example, execute the image segmentation method provided in the present disclosure.

[0065] The disclosed embodiments can be applied to scenarios where image segmentation is required, for example, image segmentation in single-frame shooting mode, image segmentation in multi-frame shooting mode, image segmentation in preview mode, image segmentation in video recording mode, image segmentation in night scene shooting or video mode, etc. For another example, the ISP module can be applied as a key module affecting the processor, including but not limited to the following scenarios:

[0066] Image segmentation in single-frame shooting mode, image segmentation in multi-frame shooting mode, image segmentation in preview mode, image segmentation in video recording mode, night scene shooting, image segmentation in video mode, etc.

[0067] Figure 2 The following is a flow chart of an image segmentation method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0068] Step 201, based on the image size of the image to be segmented and the preset number of superpixels, determining the initial cluster center corresponding to each superpixel; the preset number of superpixels is at least two;

[0069] In one embodiment, the image to be segmented is an image in RGB format. Optionally, the image to be processed may also be an image in RGB format converted from a grayscale image, a RAW format image, or the like.

[0070] In one embodiment, each superpixel corresponds to an initial cluster center. Therefore, the number of initial cluster centers is the same as the number of preset superpixels. Based on the number of initial cluster centers, all initial cluster centers can be evenly distributed on the image to be segmented. For example, if the size of the image to be segmented is 10*10, the number of preset superpixels is 10, and the number of initial cluster centers is also 10, then the step length between the initial cluster centers is the square root of (10*10 / 10), and then the initial cluster centers are evenly set on the image to be segmented based on the step length.

[0071] In one embodiment, if the number of preset superpixels is one, that is, the entire image to be segmented is one superpixel, then image segmentation will lose its meaning. Therefore, the number of preset superpixels is at least two.

[0072] Step 202, determining a pixel point with the smallest gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels; the first cluster center refers to a cluster center after the initial cluster center of each superpixel is updated;

[0073] In one embodiment, the size of the preset first neighborhood may be 2*2, 3*3, 4*4, etc., which is not limited in the present disclosure. The smaller the size of the preset first neighborhood is, the fewer scales corresponding to the pixel points that need to be acquired are, and the amount of calculation is also less. However, at the same time, due to the small size of the preset first neighborhood, the first cluster center may also be the noise or edge area in the image to be segmented. In practical applications, the size of the preset first neighborhood can be set as needed, which is not limited in the present disclosure.

[0074] In one embodiment, the pixel point with the smallest gradient value indicates that the pixel value of the image to be segmented changes slowly at this pixel point, that is, the probability that this pixel point is a noise point or an image edge pixel point is small. Further, using this pixel point as the first cluster center can obtain a better image segmentation result.

[0075] Step 203, determining a second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and spatial distance between each pixel in the image to be segmented and each of the first cluster centers;

[0076] In one embodiment, the RGB color distance and spatial distance between each pixel point in the image to be segmented and each of the first cluster centers can be calculated, and the first cluster center with the smallest sum of the RGB color distance and the spatial distance is used as the second cluster center of the corresponding pixel point in the image to be segmented.

[0077] In one embodiment, the RGB color distance and spatial distance between each pixel point in the image to be segmented and each of the first cluster centers can be calculated, and the first cluster center with the smallest weighted sum of the RGB color distance and the spatial distance is used as the second cluster center of the corresponding pixel point in the image to be segmented.

[0078] In one embodiment, the RGB color distance and spatial distance between each pixel point in the image to be segmented and each of the first cluster centers can be calculated, and the first cluster center with the smallest weighted Euclidean distance of the RGB color distance and the spatial distance is used as the second cluster center of the corresponding pixel point in the image to be segmented.

[0079] The embodiments of the present disclosure are not limited to this.

[0080] In one embodiment, optionally, the RGB color distance may be replaced by the LAB color distance. If the RGB color distance is replaced by the LAB color distance, before step 201, the image segmentation method further includes:

[0081] Converting the image to be segmented into an XYZ space image;

[0082] The XYZ space image is converted into a LAB space image.

[0083] In one embodiment, LAB is composed of brightness L and color elements a and b related to brightness. The value range of L is from 0 (black) to 100 (white). a represents the range from magenta to green, and the value range is 127 to -128 (a is negative for green, and a is positive for magenta), and b represents the range from yellow to blue, and the value range is 127 to -128 (b is negative for blue, and b is positive for yellow).

[0084] LAB space images have the following characteristics:

[0085] The L component of the LAB space image closely matches the brightness perception of the human eye, and can accurately balance the color by modifying the output of the a and b components, or use the L component to adjust the brightness. The LAB space image is a color system based on physiological characteristics and is device-independent.

[0086] The color gamut of LAB space images is wide. It not only includes all the color gamuts of RGB images and CMYK images, but also can express colors that RGB images and CMYK images cannot express. All colors that the human eye can perceive can be expressed through the Lab model.

[0087] In one embodiment, since the RGB color distance is replaced by the LAB color distance, when performing image segmentation on the image to be segmented, the image to be segmented must be converted into a LAB space image, which will greatly increase the amount of calculation and the corresponding hardware overhead cost will also increase accordingly.

[0088] Step 204: obtaining a superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented.

[0089] In one embodiment, after determining the second cluster center corresponding to each pixel point in the image to be segmented, all pixel points corresponding to the same second cluster center can be divided into a superpixel using contour lines to obtain a superpixel segmentation result of the image to be segmented.

[0090] The image segmentation method provided by the present disclosure includes:

[0091] Based on the image size of the image to be segmented and the preset number of superpixels, determine the initial cluster center corresponding to each superpixel; the preset number of superpixels is at least two; determine the pixel with the smallest gradient value in the preset first neighborhood of each initial cluster center as the first cluster center corresponding to each superpixel; the first cluster center refers to the cluster center after the initial cluster center of each superpixel is updated; based on the RGB color distance and spatial distance between each pixel in the image to be segmented and each of the first cluster centers, determine the second cluster center corresponding to each pixel in the image to be segmented; based on the second cluster center corresponding to each pixel in the image to be segmented, obtain the superpixel segmentation result of the image to be segmented.

[0092] Compared with the Simple Linear Iterative Clustering (SLIC) algorithm used in the related art, the solution provided by the present disclosure uses RGB color distance instead of LAB color distance to determine the cluster center of each pixel in the pixels to be segmented. Since the image to be processed is an RGB image, it is not necessary to convert the image to be processed into a LAB space image, which greatly reduces the amount of calculation and saves hardware costs. At the same time, compared with the image segmentation method in the related art, the solution provided by the present disclosure only needs to set the parameters of the number of superpixels to obtain better image segmentation results, further reducing the consumption of hardware resources.

[0093] In one embodiment, since the spatial distance between some pixels in the image to be segmented and some first cluster centers is far, for example, the spatial distance between the pixel located in the lower right corner of the image to be segmented and the first cluster center located in the upper left corner of the segmented image is significantly farther than the spatial distances between the first cluster centers at other positions, at this time, if the RGB color distance and spatial distance between the two are calculated again, it is an invalid calculation.

[0094] In one embodiment, in order to reduce invalid calculations and thus reduce hardware overhead, the second neighborhood size may be preset. The first cluster center takes its own spatial position as the center and calculates the RGB color distance and spatial distance between it and each pixel point in the preset second neighborhood.

[0095] Based on this, in one embodiment, step 203 may include the following steps:

[0096] Step 301, based on each of the first cluster centers and a preset second neighborhood size, determining a preset second neighborhood corresponding to each of the first cluster centers; the size of the preset second neighborhood is larger than the size of the preset superpixel;

[0097] In one embodiment, if the size of the preset superpixel is s*s, the size of the preset second neighborhood can be 2s*2s, 3s*3s, 1.5s*1.5s, etc., that is, the size of the preset second neighborhood is larger than the size of the preset superpixel. When the size of the preset second neighborhood is larger than the size of the preset superpixel, it can not only ensure that each pixel point in the image to be processed falls into the preset second neighborhood corresponding to the first cluster center, but also effectively reduce the amount of calculation of RGB color distance and spatial distance, and reduce hardware overhead.

[0098] In one embodiment, the size of the preset superpixel is the ratio of the size of the image to be processed to the number of the preset superpixels.

[0099] In one embodiment, the preset second neighborhood corresponding to the first cluster center refers to a region centered on the first cluster center and having a size equal to the size of the preset second neighborhood.

[0100] Step 302, based on the preset second neighborhood corresponding to each of the first cluster centers, determining the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood;

[0101] In one embodiment, the RGB color distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood is calculated by taking one pixel point X in each of the preset second neighborhoods as the distance. i , and one of the first cluster centers X in the corresponding preset second neighborhood k RGB color distance d RGB It can be the Euclidean distance in RGB color space, that is:

[0102]

[0103] d RGB It can also be

[0104] d RGB =|R i -R k |+|G i -G k |+|B i -B k |;

[0105] In order to reduce the amount of calculation and hardware overhead, the RGB color distance d RGB The calculation method can be the second one mentioned above.

[0106] In one embodiment, the spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood is calculated by taking one pixel point X in each of the preset second neighborhoods as i , and one of the first cluster centers X in the corresponding preset second neighborhood k The spatial distance d xy Can be

[0107]

[0108] d xy It can also be

[0109] d xy =|x i -x k |+|y i-y k |;

[0110] In order to reduce the amount of calculation and hardware overhead, the spatial distance d xy The calculation method can be the second one mentioned above.

[0111] Step 303, based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, determine the second cluster center corresponding to each pixel point in the image to be segmented.

[0112] In one embodiment, each pixel point corresponding to the same second cluster center has a high degree of similarity in features. In the image, the RGB color distance and spatial distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood can be used to determine the clustering distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood, and the first cluster center with the smallest clustering distance with each pixel point in the image to be segmented is determined as the second cluster center corresponding to each pixel point in the image to be segmented.

[0113] In one embodiment, the cluster distance D s Can be

[0114] D s =d RGB +d xy ;

[0115] The cluster distance D s It can also be

[0116] D s =q1*d RGB +q2*d xy ;

[0117] Among them, q1 and q2 are the RGB color distances d RGB and spatial distance d xy The weight of .

[0118] In one embodiment, q1 and q2 can respectively select the maximum RGB color distance and the maximum spatial distance d xy , where the maximum RGB color distance, that is, the value range of q1 is 1≤q1≤40, such as q1=10. The maximum spatial distance d xy , that is, q2 is defined as the step length s. In the embodiment of the present disclosure, the values ​​of q1 and q2 are not limited.

[0119] In one embodiment, in order to improve the image segmentation effect, the clustering distance D sThe second method can be selected for calculation.

[0120] In one embodiment, based on the above description, determining the second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and spatial distance between each pixel in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood includes:

[0121] Determine the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood;

[0122] Based on the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood, the first clustering center with the smallest clustering distance to each pixel point in the image to be segmented is determined as the second clustering center corresponding to each pixel point in the image to be segmented.

[0123] In one embodiment, based on the second clustering distance D s The calculation method, based on the LAB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, determines the cluster distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, including:

[0124] Determine the RGB color distance weight and the spatial distance weight based on the maximum spatial distance within the preset class and the maximum color distance within the preset class;

[0125] In one embodiment, the RGB color distance weight and the spatial distance weight may be represented by q1 and q2, respectively.

[0126] In one embodiment, optionally, the q1=10, and the q2=s.

[0127] In one embodiment, optionally, q1=1,

[0128] In one embodiment, the values ​​of q1 and q2 may be set based on comprehensive considerations such as calculation accuracy and hardware overhead, and are not limited in this disclosure.

[0129] Based on the RGB color distance weight, the spatial distance weight, and the RGB color distance and spatial distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood, the clustering distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood is determined.

[0130] In one embodiment, q1=1, For example:

[0131]

[0132] In one embodiment, if the RGB color distance d RGB With the LAB color distance d LAB Instead, then:

[0133]

[0134] Among them, L i , A i , B i Represents the pixel point X in the LAB space image i The pixel values ​​of L, A, and B channels, L k , A k , B k Represents the pixel point X in the LAB space image k The pixel values ​​of the L, A, and B channels.

[0135]

[0136]

[0137] By using the LAB color distance d LAB Instead of the RGB color distance d RGB Calculate D s It can be seen from the process that although it is also possible to set only one parameter, the number of preset superpixels, the amount of calculation is obviously larger and the hardware cost is higher.

[0138] In the above embodiment, the minimum clustering distance D is used s A cluster center is determined for each pixel in the image to be segmented, and superpixels with the same number as the preset superpixels are obtained, thereby achieving image segmentation. However, the cluster centers in the above embodiment are obtained based on the initial cluster centers, and the initial cluster centers are evenly distributed on the image to be segmented according to the preset number of superpixels. Therefore, the image segmentation result may not be accurate enough. Therefore, the first cluster center can also be updated in an iterative manner to further determine the cluster center of each pixel in the image to be segmented.

[0139] Based on this, in one embodiment, before obtaining the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented, the method further includes:

[0140] Using the second cluster center to update the initial cluster center, iterate, and obtain the second cluster center corresponding to each pixel in the image to be segmented after each iteration;

[0141] In one embodiment, the above iterative process refers to repeating step 202 and step 203.

[0142] In one embodiment, the spatial position of the updated initial cluster center is different from that of the initial cluster center before the update, and the pixels contained in the preset first neighborhood are also different, so the position of the updated first cluster center determined by the updated initial cluster center and the preset first neighborhood may also change. Then, the possibility that the updated first cluster center is a noise point or an image edge point of the image to be segmented is smaller, and the image segmentation result is more reliable.

[0143] In one embodiment, the initial cluster center may be updated by other methods, and then the steps 202 and 203 are repeated for iteration. For example, based on the superpixel determined last time, the initial cluster center is updated by using the pixel points corresponding to the center position of each superpixel. The center position of each superpixel refers to the pixel points corresponding to the center positions of the leftmost and rightmost pixels, or the pixel points corresponding to the center positions of the topmost and bottommost pixels, or the horizontal coordinates of the center positions of the leftmost and rightmost pixels are used as the horizontal coordinates corresponding to the center position of the corresponding superpixel, and the vertical coordinates of the center positions of the topmost and bottommost pixels are used as the vertical coordinates corresponding to the center position of the corresponding superpixel.

[0144] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met.

[0145] In one embodiment, the iteration termination condition refers to a condition corresponding to the number of iteration terminations.

[0146] In one embodiment, the iteration termination condition may be that the second cluster center corresponding to each pixel in the image to be processed no longer changes.

[0147] In one embodiment, the iteration termination condition may also be that the position of the updated initial cluster center no longer changes or the distance of its change is within a preset error distance.

[0148] In one embodiment, the iteration termination condition may also be that the number of iterations reaches a preset number of iteration terminations, which may be 8, 10, 20, etc., and is not limited in the present disclosure.

[0149] In one embodiment, if the iteration termination condition is that the number of iterations reaches a preset number of iteration terminations, then determining the second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is reached includes:

[0150] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the number of iterations reaches a preset number of iteration terminations.

[0151] In one embodiment, after the above iterations, the image to be segmented can be segmented into multiple superpixels, and the features of all pixels in each superpixel are highly uniform. Subsequent processing of the image to be segmented using superpixel dimensions instead of pixel dimensions can effectively reduce the amount of data for subsequent processing.

[0152] In one embodiment, the segmentation result of the image to be segmented during the above-mentioned iterative process may have the following defects, such as multiple connectivity of a superpixel, too small size of a superpixel, etc. At this time, in order to improve the image segmentation quality, the above defects can be eliminated by connected domain enhancement processing.

[0153] Based on this, in one embodiment, the image segmentation method further includes performing connected domain enhancement processing on the superpixel segmentation result of the image to be segmented, so as to obtain the superpixel segmentation result of the image to be segmented after the connected domain enhancement.

[0154] In one embodiment, the connected domain enhancement processing can be implemented in OpenCV, MATLAB, etc. For example, the Enforce Lable Connectivity function in OpenCV is used. The process of the Enforce Lable Connectivity function performing connected domain enhancement processing on the superpixel segmentation result of the image to be segmented is: using a marker table of a preset size, in which all element values ​​are -1, to traverse each superpixel in a Z-shaped order from left to right and from top to bottom, and reallocate discontinuous superpixels or superpixels that are too small to adjacent superpixels.

[0155] The image segmentation method provided by the present disclosure is further illustrated by taking an application example as follows.

[0156] like Figure 4 As shown, the image segmentation method provided by the first application example of the present disclosure includes the following steps:

[0157] Step 401, based on the image size of the image to be segmented and the preset number of superpixels, determining the initial cluster center corresponding to each superpixel; the preset number of superpixels is at least two;

[0158] Step 402, determining a pixel point with a minimum gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels;

[0159] Step 403: determining a preset second neighborhood corresponding to each of the first cluster centers based on each of the first cluster centers and a preset second neighborhood size;

[0160] Step 404, based on the preset second neighborhood corresponding to each of the first cluster centers, determining the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood;

[0161] Step 405, determining the second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and spatial distance between each pixel in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood;

[0162] Step 406, using the second cluster center to update the initial cluster center, repeating steps 501 to 505, performing iterations, and obtaining the second cluster center corresponding to each pixel in the image to be segmented after each iteration;

[0163] Step 407, determining a second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met;

[0164] Step 408: Obtain a superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented.

[0165] The image segmentation method provided by the present disclosure is further illustrated below with another application example.

[0166] In the second application example provided by the present disclosure, the following image segmentation architecture is used to perform image segmentation on the image to be segmented.

[0167] like Figure 5 As shown, the image segmentation architecture provided by the second application example of the present disclosure includes an input image module, an initialization module, a superpixel segmentation module and a superpixel output module.

[0168] Among them, the output end of the input image module is connected to the input ends of the initialization module and the superpixel segmentation module; the output end of the superpixel segmentation module is connected to the input end of the superpixel output module; the output end of the superpixel output module is connected to the input end of the lower-level image processing architecture, so as to provide the lower-level image processing mechanism with a superpixel segmented image of the image to be segmented.

[0169] The image input module is used to obtain the image to be segmented. In one embodiment, the image input module can use a data interface device to obtain the image to be segmented from an external image acquisition device such as a camera, a network device such as a computer, or a physical storage device.

[0170] The initialization module is used to obtain the initial cluster center based on the image information of the image to be segmented and the number of preset superpixels.

[0171] The superpixel segmentation module is used to segment the image to be segmented into multiple superpixels.

[0172] In one embodiment, the superpixel segmentation module includes an image data buffer module, a superpixel calculation module and a superpixel storage module. In the second application example of the present disclosure, the output end of the superpixel calculation module serves as the output end of the superpixel segmentation module, and the input end of the image data buffer module serves as the input end of the superpixel segmentation module. Among them, the image data buffer module is used to buffer the image information of the image to be segmented, that is, the coordinate information and pixel value information of each pixel point in the image to be segmented. The superpixel calculation module is used to perform superpixel segmentation on the image to be segmented based on the image information in the image data buffer module and the initial cluster center information in the initialization module; the superpixel storage module is used to store superpixel information.

[0173] Based on the image segmentation architecture provided by the application example of the present disclosure, the application example of the present disclosure provides an image segmentation method, such as Figure 6 As shown, the following steps are included:

[0174] Step 601, obtaining the image to be segmented through the image input module;

[0175] Wherein, the image to be segmented is an RGB image.

[0176] Step 602, temporarily storing the image information of the image to be segmented by the first image data buffer module;

[0177] Step 603, based on the image to be segmented and the number of preset superpixels, determine the initial cluster center through the initialization module;

[0178] The number of the initial cluster centers is the same as the number of preset superpixels.

[0179] Since the initial cluster center is determined based on the number of superpixels preset in the initialization module, there is no need to wait for superpixel information, so the delay time of the initialization module is short.

[0180] Step 604, based on the initial cluster center and the image information of the image to be segmented, determine the second cluster center corresponding to each pixel in the image to be segmented by the superpixel calculation module;

[0181] In the second application example of the present disclosure, the process of the superpixel calculation module determining the second cluster center corresponding to each pixel point in the image to be segmented is the process from step 202 to step 203 above, which will not be described in detail here.

[0182] In the second application example of the present disclosure, the process of the superpixel calculation module determining the second cluster center corresponding to each pixel point in the image to be segmented may also be as follows:

[0183] Traverse each pixel point in the image to be processed, calculate the clustering distance between each pixel point in the image to be processed and each of the initial clustering centers, and determine the initial clustering center and pixel point with the smallest clustering distance as the second clustering center corresponding to the corresponding pixel point in the image to be segmented.

[0184] Step 605, storing the second cluster center corresponding to each pixel point in the image to be segmented determined by the first superpixel calculation module through the superpixel storage module;

[0185] Step 606, using the mean of all pixel points corresponding to each second cluster center to update the initial cluster center, and repeating the above steps 604 and 605 to perform iteration;

[0186] It should be noted that the mean of all pixels corresponding to the second cluster center includes the spatial position mean and the pixel value mean, the horizontal coordinate mean of the spatial position can be the horizontal coordinate mean of all pixels corresponding to the second cluster center, the vertical coordinate mean of the spatial position can be the vertical coordinate mean of all pixels corresponding to the second cluster center, and the pixel value mean can be the mean of the three-channel pixel values ​​of the pixels corresponding to the spatial position mean. Because the image to be segmented is an RGB three-channel image, the image information in the image data buffer module is also stored according to the three channels of R, G, and B, and contains the coordinate information corresponding to each pixel.

[0187] In the above iterative process, taking the preset number of superpixels as 9 as an example, the superpixel storage module can be a 3*3 storage structure; wherein each storage structure contains a second cluster center information and all corresponding pixel point information, and all the pixel points contained therein can be labeled with the second cluster center.

[0188] When the initial cluster center is updated, each column in the 3*3 storage structure can be moved one column to the left, and the information of the updated initial cluster center can be stored in the third column. When traversing each pixel point in the image to be processed, the cluster distance between each pixel point in the image to be processed and each initial cluster center is calculated, and the initial cluster center and pixel point with the smallest cluster distance are determined as the second cluster center corresponding to the corresponding pixel point in the image to be segmented, and then the corresponding pixel point is labeled with the corresponding second cluster center. When the initial cluster center is updated, each row in the 3*3 storage structure can be moved one column upward, and the information of the updated initial cluster center can be stored in the third row.

[0189] Step 607: The superpixel output module outputs the second cluster center corresponding to each pixel point in the image to be segmented.

[0190] like Figure 7 As shown, the embodiment of the present disclosure further provides an image segmentation device, the image segmentation device 700 comprising:

[0191] An initial cluster center determination unit 701 is used to determine the initial cluster center corresponding to each superpixel based on the image size of the image to be segmented and a preset number of superpixels; the preset number of superpixels is at least two;

[0192] A first cluster center determination unit 702 is used to determine a pixel point with the smallest gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels; the first cluster center refers to a cluster center after the initial cluster center of each superpixel is updated;

[0193] A second cluster center determination unit 703 is used to determine a second cluster center corresponding to each pixel point in the image to be segmented based on the RGB color distance and the spatial distance between each pixel point in the image to be segmented and each of the first cluster centers;

[0194] The superpixel segmentation result acquisition unit 704 is used to acquire the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented.

[0195] In one embodiment, the second cluster center determination unit 703 is specifically configured to:

[0196] Based on each of the first cluster centers and a preset second neighborhood size, determining a preset second neighborhood corresponding to each of the first cluster centers; the size of the preset second neighborhood is larger than the size of the preset superpixel;

[0197] Based on the preset second neighborhood corresponding to each of the first cluster centers, determine the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood;

[0198] Based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, the second cluster center corresponding to each pixel point in the image to be segmented is determined.

[0199] In one embodiment, the second cluster center determination unit 703 is specifically configured to:

[0200] Determine the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood;

[0201] Based on the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood, the first clustering center with the smallest clustering distance to each pixel point in the image to be segmented is determined as the second clustering center corresponding to each pixel point in the image to be segmented.

[0202] In one embodiment, the second cluster center determination unit 703 is specifically configured to:

[0203] Determine the RGB color distance weight and the spatial distance weight based on the maximum spatial distance within the preset class and the maximum color distance within the preset class;

[0204] Based on the RGB color distance weight, the spatial distance weight, and the RGB color distance and spatial distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood, the clustering distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood is determined.

[0205] In one embodiment, the image segmentation apparatus 700 further includes an iteration unit, and the iteration unit is used to:

[0206] Using the second cluster center to update the initial cluster center, iterate, and obtain the second cluster center corresponding to each pixel in the image to be segmented after each iteration;

[0207] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met.

[0208] In one embodiment, the iteration unit is specifically used for:

[0209] Determine a second cluster center corresponding to each pixel point in the image to be segmented after the number of iterations reaches a preset number of iteration terminations.

[0210] In one embodiment, the image segmentation apparatus 700 further includes a connected domain enhancement unit, and the connected domain enhancement unit is used to:

[0211] Performing connected domain enhancement processing on the superpixel segmentation result of the image to be segmented, to obtain the superpixel segmentation result of the image to be segmented after the connected domain enhancement.

[0212] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0213] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0214] Figure 8 8 is a block diagram of an electronic device 800 for implementing the above-mentioned image segmentation method according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0215] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0216] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0217] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0218] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0219] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0220] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0221] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0222] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0223] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0224] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0225] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0226] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image segmentation method described in the above embodiments of the present disclosure.

[0227] The embodiment of the present disclosure also provides a chip, which can be seen in Fig. 9 Schematic diagram of the chip structure shown. Fig. 9 The chip shown includes a processor 901 and an interface 902. The number of processors 901 can be one or more, and the number of interfaces 902 can be multiple. The interface 902 is used to receive a signal from a memory of an electronic device and send the signal to the processor 901, wherein the signal includes a computer instruction stored in the memory.

[0228] Optionally, the chip further includes a memory 903, and the memory 903 is used to store necessary computer instructions and data. When the processor 901 executes the computer instructions, the electronic device executes the image segmentation method described in the above embodiment of the present disclosure.

[0229] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0230] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0231] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0232] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0233] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0234] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0235] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0236] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image segmentation method, characterized in that: include: Based on the image size of the image to be segmented and the preset number of superpixels, determining an initial cluster center corresponding to each superpixel; The preset number of superpixels is at least two; Determine a pixel point with the smallest gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels; The first cluster center refers to the cluster center after the initial cluster center of each superpixel is updated; Determine a second cluster center corresponding to each pixel in the image to be segmented based on the RGB color distance and the spatial distance between each pixel in the image to be segmented and each of the first cluster centers; Based on the second cluster center corresponding to each pixel point in the image to be segmented, a superpixel segmentation result of the image to be segmented is obtained.

2. The method according to claim 1, characterized in that The determining, based on the RGB color distance and the spatial distance between each pixel in the image to be segmented and each of the first cluster centers, the second cluster center corresponding to each pixel in the image to be segmented comprises: Based on each of the first cluster centers and a preset second neighborhood size, determining a preset second neighborhood corresponding to each of the first cluster centers; the size of the preset second neighborhood is larger than the size of the preset superpixel; Based on the preset second neighborhood corresponding to each of the first cluster centers, determine the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood; Based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first cluster center in the corresponding preset second neighborhood, the second cluster center corresponding to each pixel point in the image to be segmented is determined.

3. The method according to claim 2, characterized in that The determining of the second cluster center corresponding to each pixel point in the image to be segmented based on the RGB color distance and the spatial distance between each pixel point in each preset second neighborhood and each first cluster center in the corresponding preset second neighborhood includes: Determine the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the RGB color distance and spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood; Based on the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood, the first clustering center with the smallest clustering distance to each pixel point in the image to be segmented is determined as the second clustering center corresponding to each pixel point in the image to be segmented.

4. The method according to claim 3, characterized in that: The step of determining the clustering distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood based on the LAB color distance and the spatial distance between each pixel point in each of the preset second neighborhoods and each first clustering center in the corresponding preset second neighborhood includes: Determine the RGB color distance weight and the spatial distance weight based on the maximum spatial distance within the preset class and the maximum color distance within the preset class; Based on the RGB color distance weight, the spatial distance weight, and the RGB color distance and spatial distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood, the clustering distance between each pixel point in the preset second neighborhood and each first cluster center in the corresponding preset second neighborhood is determined.

5. The method according to any one of claims 1 to 4, characterized in that Before obtaining the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented, the method further includes: Using the second cluster center to update the initial cluster center, iterate, and obtain the second cluster center corresponding to each pixel in the image to be segmented after each iteration; Determine a second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met.

6. The method according to claim 5, characterized in that The determining of the second cluster center corresponding to each pixel point in the image to be segmented after the iteration termination condition is met includes: Determine a second cluster center corresponding to each pixel point in the image to be segmented after the number of iterations reaches a preset number of iteration terminations.

7. The method according to claim 6, characterized in that The method further comprises: Performing connected domain enhancement processing on the superpixel segmentation result of the image to be segmented, to obtain the superpixel segmentation result of the image to be segmented after the connected domain enhancement.

8. An image segmentation device, characterized in that: include: An initial cluster center determination unit, configured to determine an initial cluster center corresponding to each superpixel based on the image size of the image to be segmented and a preset number of superpixels; The preset number of superpixels is at least two; A first cluster center determination unit, used to determine a pixel point with a minimum gradient value in a preset first neighborhood of each of the initial cluster centers as a first cluster center corresponding to each of the superpixels; The first cluster center refers to the cluster center after the initial cluster center of each superpixel is updated; A second cluster center determination unit, configured to determine a second cluster center corresponding to each pixel point in the image to be segmented based on an RGB color distance and a spatial distance between each pixel point in the image to be segmented and each of the first cluster centers; The superpixel segmentation result acquisition unit is used to acquire the superpixel segmentation result of the image to be segmented based on the second cluster center corresponding to each pixel point in the image to be segmented.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

11. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 7.

Images