Image segmentation method and electronic device

Abstract

The present invention provides an image segmentation method and an electronic device. The image segmentation method includes: obtaining a first grayscale image; performing regression analysis on the first grayscale image to obtain a residual image and determining an object backbone region of the residual image; calculating an average grayscale value of the object backbone region in the residual image to define the pixel value of each pixel in the object backbone region as the average grayscale value and generating a second grayscale image having the object backbone region; recursively determining whether the residual polarities of multiple adjacent pixels of multiple edge pixels adjacent to the object backbone region in the residual image are the same as the residual polarity of the corresponding edge pixel, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than respective corresponding first thresholds to expand the object backbone region of the second grayscale image; and extracting the object backbone region of the second grayscale image after the recursive determination as a target object. The present invention can effectively extract the target object.

Description

Technical Field

[0001] The present invention relates to an image processing technology, and more particularly to an image segmentation method and an electronic device. Background Art

[0002] The image segmentation technology in digital image processing has long been a key technology in the fields of image processing and computer vision. Especially when using digital images as a medium to predict, analyze, inspect, or measure objects in a target area, the first condition is to first define the clear range of the target object in the image. However, traditional image segmentation is based on the distribution in the color space or the gray scale space, without considering uneven illumination or significant changes in the reflection brightness of the target object itself. In other words, the judgment information on which traditional image segmentation is based does not distinguish whether the pixels are from the background or the object, nor whether they are from the pixels near the boundary. Therefore, traditional image segmentation usually has a large amount of noise or the judged object range is incomplete. In view of this, how to implement effective image segmentation to increase the reliability and usability of image segmentation will be proposed with several solution examples below.

[0003] The "Background Art" paragraph is only used to help understand the content of the present invention. Therefore, the content disclosed in the "Background Art" paragraph may include some known technologies that are not known to those of ordinary skill in the art. The content disclosed in the "Background Art" paragraph does not represent the problems to be solved by the content or one or more embodiments of the present invention, which have been known or recognized by those of ordinary skill in the art before the application of the present invention. Summary of the Invention

[0004] The present invention is directed to an image segmentation method and an electronic device, which can segment the target object and the background image in the image to effectively extract the target object.

[0005] To achieve one or part or all of the above purposes or other purposes, the image segmentation method of the present invention includes the following steps: obtaining a first grayscale image; performing regression analysis on the first grayscale image to obtain a residual image, and determining the object backbone region of the residual image; calculating the average grayscale value of the object backbone region in the residual image to define the pixel value of each pixel in the object backbone region as the average grayscale value, and generating a second grayscale image having the object backbone region; recursively determining whether the residual polarities of multiple adjacent pixels adjacent to the object backbone region in the residual image are the same as the residual polarity of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than respective corresponding first thresholds, so as to define the adjacent pixels that meet the judgment as new edge pixels to expand the object backbone region in the second grayscale image; and extracting the object backbone region in the second grayscale image after the recursive judgment as a target object, wherein the step of performing the regression analysis on the first grayscale image to obtain the residual image includes: performing second-order regression analysis on the first grayscale image to remove the part of the first grayscale image having pixel values belonging to outliers; performing fourth-order regression analysis on the first grayscale image after the second-order regression analysis to estimate the background image; and subtracting the pixel value of each pixel of the first grayscale image from the pixel value of each pixel of the background image to obtain the residual image.

[0006] To achieve one or part or all of the above purposes or other purposes, the electronic device with an image segmentation function according to the present invention includes an image sensor and a processor. The image sensor is used to obtain a first grayscale image of a target area. The processor is coupled to the image sensor. The processor is used to perform the following operations: perform regression analysis on the first grayscale image to obtain a residual image, and determine the object backbone area of the residual image; calculate the average grayscale value of the object backbone area in the residual image, define the pixel value of each pixel in the object backbone area as the average grayscale value, and generate a second grayscale image with the object backbone area; recursively determine whether the residual polarities of multiple adjacent pixels of multiple edge pixels adjacent to the object backbone area in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than respective corresponding first thresholds, so as to define the adjacent pixels that meet the judgment as new edge pixels to expand the object backbone area in the second grayscale image; and extract the object backbone area in the second grayscale image after recursive judgment as the target object in the target area, wherein the operation of performing the regression analysis on the first grayscale image to obtain the residual image includes: performing second-order regression analysis on the first grayscale image to remove the part with pixel values belonging to outliers in the first grayscale image; performing fourth-order regression analysis on the first grayscale image after second-order regression analysis to estimate the background image; and subtracting the pixel value of each pixel of the first grayscale image from the pixel value of each pixel of the background image to obtain the residual image.

[0007] Based on the above, the image segmentation method and the electronic device of the present invention can perform image analysis and processing on grayscale images, so that the electronic device can automatically segment the target object in the grayscale image from the background image to effectively extract the target object in the image.

[0008] To make the above features and advantages of the present invention more obvious and understandable, specific embodiments are hereinafter given and described in detail in conjunction with the accompanying drawings as follows Description of the Drawings

[0009] The accompanying drawings are included to provide a further understanding of the present invention, and the drawings are incorporated into and constitute a part of this specification. The drawings illustrate embodiments of the present invention and, together with the description, are used to explain the principles of the present invention.

[0010] Figure 1 It is a functional block diagram of an electronic device according to an embodiment of the present invention;

[0011] Figure 2 It is a flowchart of an image segmentation method according to an embodiment of the present invention;

[0012] Figure 3 Flow chart of obtaining a residual image according to an embodiment of the present invention;

[0013] Figure 4 Flow chart of determining an object backbone region according to an embodiment of the present invention;

[0014] Figures 5A to 5E Schematic diagrams of multiple images according to an embodiment of the present invention;

[0015] Figure 6 Flow chart of re - expanding the object backbone region in the second grayscale image according to an embodiment of the present invention. Detailed implementation manners

[0016] Regarding the foregoing and other technical contents, features and effects of the present invention, they will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. Directional terms mentioned in the following embodiments, such as: up, down, left, right, front or back, etc., are only with reference to the directions of the accompanying drawings. Therefore, the directional terms used are for illustration and not for limiting the present invention.

[0017] In order to make the content of the present invention easier to understand, the following specific examples are given as examples that the present invention can actually be implemented according to. Additionally, wherever possible, elements / components / steps with the same reference numerals in the drawings and embodiments represent the same or similar parts.

[0018] Figure 1 Functional block diagram of an electronic device according to an embodiment of the present invention. Refer to Figure 1, the electronic device 100 of this embodiment has an Image Segmentation function. The electronic device 100 includes a processor 110, an image sensor 120, and a memory 130. The processor 110 is coupled to the image sensor 120 and the memory 130. In this embodiment, the memory 130 can be used to store a plurality of image processing programs for implementing the image segmentation function of the present invention and a plurality of images generated during the image segmentation process for the processor 110 to read. In this embodiment, the plurality of image processing programs are established based on the image segmentation methods described in the embodiments of the present invention, and the image segmentation methods described in the embodiments of the present invention can be implemented, for example, by extending, improving, or varying at least one of the Threshold based method, Edge based method, Region based method, Clustering method, Spectral clustering method, Watershed method, Level set method, and Neural network based method. In this embodiment, the electronic device 100 can be used, for example, in the production and manufacturing process of a display device to capture an image of the display device and perform image analysis to automatically determine whether there are unexpected objects on the display surface of the display device. In other words, the target object described in the embodiments of the present invention refers to an unexpected object automatically identified by the electronic device 100 for the image of the target area.

[0019] In this embodiment, the processor 110 includes a central processing unit (CPU) with image processing capabilities, or other programmable general-purpose or special-purpose microprocessors, image processing units (IPUs), graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), other similar arithmetic circuits, or combinations of these circuits. In this embodiment, the image sensor 120 can be a camera. In this embodiment, the memory 130 can be, for example, a dynamic random access memory (DRAM), a flash memory, or a non-volatile random access memory (NVRAM), etc.

[0020] In this embodiment, the image sensor 120 is used to obtain a first grayscale image of a target area and provide the first grayscale image to the processor 110. The processor 110 performs regression analysis on the first grayscale image to obtain a residual image and determines the object backbone area of the residual image. The processor 110 calculates the average grayscale value of the object backbone area in the residual image, defines the pixel value of each pixel in the object backbone area as the average grayscale value, and generates a second grayscale image with the object backbone area. Then, the processor 110 recursively determines whether the residual polarities of the multiple adjacent pixels of the multiple edge pixels adjacent to the object backbone area in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than the respective corresponding first thresholds to expand the object backbone area in the second grayscale image. Therefore, the processor 110 can extract the object backbone area in the recursively determined second grayscale image as the target object (object image) in the target area.

[0021] It should be noted that the above-mentioned edge pixels refer to the pixels adjacent to the non-object backbone area in the object backbone area, and the above-mentioned adjacent pixels refer to the pixels adjacent to the edge pixels of the object backbone area in the non-object backbone area. Moreover, the first threshold corresponding to each of the multiple adjacent pixels in this embodiment is the intermediate value between the pixel value of the corresponding edge pixel in the second grayscale image (i.e., the above-mentioned average grayscale value) and the pixel value of the corresponding adjacent pixel in the background image. Additionally, the regression analysis performed by the above-mentioned processor 110 refers to using the processor 110 to execute one or more specific functions to perform operations on each pixel in the image in sequence (recursively).

[0022] Figure 2 It is a flowchart of an image segmentation method according to an embodiment of the present invention. Figures 5A to 5E They are schematic diagrams of multiple images according to an embodiment of the present invention. Refer to Figure 1 、 Figure 2 and Figures 5A to 5E , the image segmentation method of this embodiment can be at least applicable to Figure 1 the electronic device 100 in the embodiment. The electronic device 100 can execute steps S210 to S250 to sequentially generate as Figures 5A to 5E . In step S210, the image sensor 120 faces the target area to obtain the first grayscale image 501 as shown in Figure 5A . In step S220, the processor 110 performs regression analysis on the first grayscale image 501 to obtain a residual image and determines the object backbone area of the residual image. In step S230, the processor 110 calculates the average grayscale value of the object backbone area in the residual image to define the pixel value of each pixel in the object backbone area as the average grayscale value and generates the second grayscale image 504 with the object backbone area MA2 as shown in Figure 5D . In step S240, the processor 110 recursively determines whether the residual polarities of the multiple adjacent pixels adjacent to the object backbone area in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than the corresponding first thresholds, so as to expand the object backbone area MA2 in the second grayscale image 504 and generate the second grayscale image 505 after recursive determination as shown in Figure 5E . In step S250, the processor 110 extracts the object backbone area MA3 in the second grayscale image 505 after recursive determination as the target object. Therefore, the image segmentation method and the electronic device 100 of this embodiment can effectively segment the target object (object image) in the image of the target area from the background image to correctly identify the position of the target object. And the step details of the above step S220 will be described by the following Figure 3 and Figure 4The process will be further described below.

[0023] Figure 3 FIG. is a flowchart of obtaining a residual image according to an embodiment of the present invention. In one embodiment, regarding step S220 of the above Figure 2 embodiment, the processor 110 may further execute steps S310 to S330 as follows Figure 3 to obtain a residual image. In step S310, the processor 110 performs a second-order regression analysis on the first grayscale image 501 to remove the part with pixel values belonging to outliers in the first grayscale image. In step S320, the processor 110 performs a fourth-order regression analysis on the first grayscale image 501 that has undergone the second-order regression analysis to estimate the background image. In step S330, the processor 110 subtracts the pixel values of each pixel of the first grayscale image from the pixel values of each pixel of the background image to obtain a residual image. It should be noted that the regression analysis of this embodiment can obtain sufficient teachings, suggestions, and implementation descriptions based on relevant image analysis techniques in the field of image processing, so it will not be elaborated here.

[0024] Figure 4 FIG. is a flowchart of determining an object backbone region according to an embodiment of the present invention. In one embodiment, regarding step S220 of the above Figure 2 embodiment, the processor 110 may further execute steps S410 to S450 as follows Figure 4 to determine the object backbone region. In step S410, the processor 110 removes the part with pixel values belonging to outliers in the residual image. In step S420, the processor 110 calculates the standard deviation based on the pixel values of multiple pixels in the remaining part of the residual image and the background image. For example, the processor 110 may divide the residual image into multiple sub-image regions, generate a grayscale histogram, and remove the larger outlier regions in the histogram, and finally calculate the standard deviation of the residual image with respect to the background image. In step S430, the processor 110 determines a second threshold based on the standard deviation. In step S440, the processor 110 determines multiple object core regions CA in the residual image 502 as shown in Figure 5B . In this embodiment, the processor 110 binarizes the absolute value of the pixel value of each pixel in the residual image according to the second threshold determined by the standard deviation to determine the object core region CA in the residual image. In step S450, recursively determine whether the residual polarities of multiple adjacent pixels of multiple edge pixels of adjacent object core regions CA in the residual image 502 are the same as the residual polarity of the corresponding edge pixels, and whether the pixel values of multiple adjacent pixels are greater than a third threshold, so as to expand the object core region CA to form an object backbone region MA1 in the image 503 as shown in Figure 5C .

[0025] It should be noted that the second threshold value described in this embodiment is obtained by multiplying the above-mentioned standard deviation by a preset gain value and adding a preset offset value. Moreover, the third threshold value described in this embodiment is obtained by multiplying the above-mentioned second threshold value by a first preset value, and the first preset value is less than 1 and greater than 0.

[0026] Figure 6 It is a flowchart for further expanding the object backbone area in the second grayscale image according to an embodiment of the present invention. Refer to Figure 2 , Figures 5A to 5E and Figure 6 , in one embodiment, steps S610 and S620 can be continued after Figure 2 step S250 to make the range of the object backbone area in the second grayscale image closer to the actual object. In step S610, the processor 110 recursively determines whether the pixel values of multiple adjacent pixels of multiple edge pixels adjacent to the object backbone area in the residual image are greater than a fourth threshold value. In step S620, the processor 110 multiplies the original pixel value of the adjacent pixel that meets the judgment in the first grayscale image 501 by a preset value, and adds the pixel value of the corresponding edge pixel (such as the edge of the object backbone area MA3) in the second grayscale image 505 multiplied by another preset value to update the pixel value of the new edge pixel.

[0027] It should be noted that the fourth threshold value described in this embodiment is obtained by multiplying the above-mentioned third threshold value by a second preset value, and the second preset value is less than 1 and greater than 0. Moreover, defining the adjacent pixel that meets the judgment as the new edge pixel means that the processor 110 multiplies the original pixel value of the adjacent pixel that meets the judgment in the first grayscale image 501 by a third preset value, and adds the pixel value of the corresponding edge pixel in the second grayscale image 505 multiplied by a fourth preset value to update the pixel value of the new edge pixel in the second grayscale image 505, where the sum of the third preset value and the fourth preset value is 1. In this regard, the new edge pixel will become the edge pixel of the new object backbone area in the second grayscale image 505. Therefore, steps S610 to S620 of this embodiment can effectively expand the range of the object backbone area MA3 in the second grayscale image 505, so that the object image generated by the processor 110 can be closer to the actual object.

[0028] In summary, the image segmentation method and the electronic device of the present invention can capture an image of a target area to obtain an image of the target area, and generate a grayscale image based on the image of the target area. Then, the image segmentation method and the electronic device of the present invention can perform image analysis and processing on the grayscale image to automatically segment the target object in the grayscale image from the background image, so as to effectively extract the target object in the image of the target area.

[0029] The above description is only a preferred embodiment of the present invention, and the scope of implementation of the present invention cannot be limited thereby. That is, all simple equivalent changes and modifications made according to the claims of the present invention and the content of the invention still fall within the scope covered by the patent of the present invention. In addition, any embodiment or claim of the present invention does not have to achieve all the purposes, advantages or features disclosed in the present invention. In addition, the abstract and the title of the invention are only used to assist in the retrieval of patent documents and are not used to limit the scope of rights of the present invention. In addition, the terms "first", "second", etc. mentioned in this specification or claims are only used to name elements or to distinguish different embodiments or scopes, and are not used to limit the upper or lower limits of the number of elements.

Claims

1. An image segmentation method, characterized in that, comprising: obtaining a first grayscale image; performing regression analysis on the first grayscale image to obtain a residual image, and determining an object backbone region of the residual image; calculating an average grayscale value of the object backbone region in the residual image, defining the pixel value of each pixel in the object backbone region as the average grayscale value, and generating a second grayscale image having the object backbone region; recursively determining whether the residual polarities of multiple adjacent pixels adjacent to the object backbone region in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels in the residual image are greater than respective first thresholds, so as to define the adjacent pixels that meet the determination as new edge pixels to expand the object backbone region in the second grayscale image; and extracting the object backbone region in the second grayscale image after recursive determination as a target object, wherein, the step of performing the regression analysis on the first grayscale image to obtain the residual image includes: performing second-order regression analysis on the first grayscale image to remove a part where the pixel values in the first grayscale image belong to outliers; performing fourth-order regression analysis on the first grayscale image after second-order regression analysis to estimate a background image; and subtracting the pixel value of each pixel of the first grayscale image from the pixel value of each pixel of the background image to obtain the residual image.

2. The image segmentation method according to claim 1, characterized in that, the respective first thresholds corresponding to the multiple adjacent pixels are intermediate values between the pixel values of the respective corresponding edge pixels in the second grayscale image and the pixel values of the respective corresponding adjacent pixels in the background image.

3. The image segmentation method according to claim 1, characterized in that, the step of determining the object backbone region of the residual image includes: removing a part where the pixel values in the residual image belong to outliers; calculating a standard deviation according to the multiple pixel values of the remaining part of the residual image and the background image; determining a second threshold according to the standard deviation; determining an object core region in the residual image according to the second threshold; and recursively determining whether the residual polarities of multiple adjacent pixels adjacent to the object core region in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the multiple adjacent pixels are greater than a third threshold, so as to expand the object core region to form the object backbone region.

4. The image segmentation method according to claim 3, characterized in that, the step of determining the second threshold according to the standard deviation includes: multiplying the standard deviation by a preset gain value and adding a preset offset value to obtain the second threshold.

5. The image segmentation method according to claim 4, characterized in that, the third threshold is the second threshold multiplied by a first preset value, and the first preset value is less than 1 and greater than 0.

6. The image segmentation method according to claim 3, wherein, the step of determining the object core region in the residual image according to the second threshold includes: performing binarization on the absolute value of the pixel value of each of the multiple pixels in the residual image according to the second threshold to determine the object core region in the residual image.

7. The image segmentation method according to claim 3, wherein, the step of recursively determining the multiple adjacent pixels of the multiple edge pixels adjacent to the object backbone region in the residual image further includes: recursively determining whether the pixel value of each of the multiple adjacent pixels of the multiple edge pixels adjacent to the object backbone region in the residual image is greater than a fourth threshold, and defining the adjacent pixels that meet the determination as new edge pixels to expand the object backbone region in the second grayscale image.

8. The image segmentation method according to claim 7, wherein, the fourth threshold is the third threshold multiplied by a second preset value, and the second preset value is less than 1 and greater than 0.

9. The image segmentation method according to claim 7, wherein, the step of defining the adjacent pixels that meet the determination as the new edge pixels includes: multiplying the original pixel value of the adjacent pixels that meet the determination in the first grayscale image by a third preset value, and adding the pixel value of the corresponding edge pixel in the second grayscale image multiplied by a fourth preset value to update the pixel value of the new edge pixel, wherein the sum of the third preset value and the fourth preset value is 1.

10. An electronic device having an image segmentation function, wherein, it includes an image sensor and a processor, wherein, the image sensor is used to obtain a first grayscale image towards a target area; and the processor is coupled to the image sensor and is used to perform the following operations: performing regression analysis on the first grayscale image to obtain a residual image, and determining the object backbone region of the residual image; calculating the average grayscale value of the object backbone region in the residual image to define the pixel value of each pixel in the object backbone region as the average grayscale value, and generating a second grayscale image having the object backbone region; recursively determining whether the residual polarities of the multiple adjacent pixels of the multiple edge pixels adjacent to the object backbone region in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel value of each of the multiple adjacent pixels in the residual image is greater than the corresponding first threshold, and defining the adjacent pixels that meet the determination as new edge pixels to expand the object backbone region in the second grayscale image; and extracting the object backbone region in the second grayscale image after recursive determination as the target object in the target area, wherein, the operation of performing the regression analysis on the first grayscale image to obtain the residual image includes: performing second-order regression analysis on the first grayscale image to remove the part of the first grayscale image having pixel values belonging to outliers; Perform a fourth-order regression analysis on the first grayscale image that has undergone a second-order regression analysis to estimate the background image; and Subtract the pixel values of each pixel of the first grayscale image from the pixel values of each pixel of the background image to obtain the residual image.

11. The electronic device according to claim 10,[[]] wherein,[[]] the first threshold corresponding to each of the plurality of adjacent pixels is the intermediate value between the pixel value of the corresponding edge pixel in the second grayscale image and the pixel value of the corresponding adjacent pixel in the background image.

12. The electronic device according to claim 10,[[]] wherein,[[]] the operation of determining the object backbone region of the residual image includes:[[]] Removing the part in the residual image that has pixel values belonging to outliers; Calculating the standard deviation based on the pixel values of the plurality of pixels in the remaining part of the residual image and the background image; Determining a second threshold based on the standard deviation; Determining the object core region in the residual image based on the second threshold; and Recursively determining whether the residual polarities of the plurality of adjacent pixels adjacent to the object core region in the residual image are the same as the residual polarities of the corresponding edge pixels, and whether the pixel values of the plurality of adjacent pixels are greater than a third threshold, to expand the object core region to form the object backbone region.

13. The electronic device according to claim 12,[[]] wherein,[[]] the operation of determining the second threshold based on the standard deviation includes:[[]] Multiplying the standard deviation by a preset gain value and adding a preset offset value to obtain the second threshold.

14. The electronic device according to claim 13,[[]] wherein,[[]] the third threshold is the second threshold multiplied by a first preset value, and the first preset value is less than 1 and greater than 0.

15. The electronic device according to claim 12,[[]] wherein,[[]] the operation of determining the object core region in the residual image based on the second threshold includes:[[]] Binarizing the absolute values of the pixel values of the plurality of pixels in the residual image based on the second threshold to determine the object core region in the residual image.

16. The electronic device according to claim 12,[[]] wherein,[[]] the operation of recursively determining the plurality of adjacent pixels of the plurality of edge pixels adjacent to the object backbone region in the residual image further includes:[[]] Recursively determining whether the pixel values of the plurality of adjacent pixels of the plurality of edge pixels adjacent to the object backbone region in the residual image are greater than a fourth threshold, to define the adjacent pixels that meet the determination as new edge pixels, to expand the object backbone region in the second grayscale image.

Images