Image segmentation method, device, electronic device and computer storage medium

By acquiring and processing the pixel grayscale histogram of underwater images, adaptively adjusting the threshold, the problems of uneven light and low contrast of underwater images are solved, improving segmentation accuracy and reducing the blocking effect.

CN114240989BActive Publication Date: 2024-11-26INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111453436.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-26
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

Due to uneven light and low contrast, underwater images lead to the blocking effect of traditional segmentation methods, low segmentation accuracy, and lack of effective solutions.

Method used

By acquiring the pixel grayscale histogram of the target image, determining the segmentation threshold, and setting the first and second pixel values ​​on the basis of the pixel points, generating the compensated image by traversing the area, combining the EM algorithm and the Otsu method, adaptive adjustment Thresholds to improve segmentation accuracy.

Benefits of technology

It improves the accuracy of underwater image segmentation, reduces the blocking effect, and adapts to image segmentation under different lighting and turbid conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114240989B_ABST
    Figure CN114240989B_ABST
Patent Text Reader

Abstract

The present invention discloses an image segmentation method, device, electronic device and computer storage medium, which relate to the field of artificial intelligence image recognition. The method comprises obtaining a pixel grayscale histogram of a target area of ​​a target image; determining a corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; setting the central pixel point to a first pixel value when the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold; setting the central pixel point to a second pixel value when the pixel value of the central pixel point of the target area is less than the segmentation threshold, and the pixel difference between the first pixel value and the second pixel value is a preset threshold; determining that multiple areas correspond to the first pixel value or the second pixel value respectively, and generating a target image segmentation image. The present invention solves the technical problem in the related art that the image segmentation accuracy is low due to uneven illumination of underwater segmented images and the existence of a blocking effect in traditional segmentation methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image segmentation method, device, electronic equipment and computer storage medium. Background Art

[0002] Compared with normal images, underwater images have the following problems: uneven illumination distribution and low contrast. Uneven illumination distribution is mainly caused by fill light. Since the fill light LEDs are concentrated around the camera, the captured images basically have a bright center, and the light decays from the center to the surroundings. The grayscale value at the edges is too low, resulting in the feed particles at the edges being extremely unclear, which brings great difficulties to image processing. In addition, the contrast of images varies greatly in water with different turbidity. The contrast is large in clear water and too small in turbid water. In actual situations, the water in scenes (fish ponds or shrimp ponds) that require underwater image segmentation is generally turbid, which results in the contrast of the captured video being too low, and it is difficult to distinguish between foreground objects and background objects. This feature is also a major problem in image processing.

[0003] Due to the uneven underwater illumination, the traditional global threshold segmentation method is not applicable to underwater image processing. The local threshold segmentation method, which divides the entire image into several small sub-images and then applies the global threshold segmentation method to each sub-image, has a serious blocking effect due to the discontinuity of the grayscale changes of adjacent sub-images, that is, the jump of the threshold at the boundary of adjacent sub-images, resulting in low image segmentation accuracy.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide an image segmentation method, device, electronic device and computer storage medium to at least solve the technical problem in the related art that the image segmentation accuracy is low due to uneven illumination of underwater segmented images and the existence of blocking effect in traditional segmentation methods.

[0006] According to one aspect of an embodiment of the present invention, there is provided an image segmentation method, comprising: obtaining a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of a plurality of areas of the target image, and central pixel points of the plurality of areas respectively correspond to pixel points of the target image; determining a corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; setting the central pixel point to a first pixel value when the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold; setting the central pixel point to a second pixel value when the pixel value of the central pixel point of the target area is less than the segmentation threshold, to obtain a compensated target area, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range; determining compensated areas corresponding to the plurality of areas respectively by traversing the plurality of areas, and generating a segmented image of the target image by combining all enhanced areas of the target image.

[0007] Optionally, obtaining a pixel grayscale histogram of a target area of ​​a target image includes: subtracting one from half of a preset size of a mask window to obtain an extended size for expanding the target image, wherein the mask window is square; expanding the target image according to the extended size to obtain an extended image after expansion, wherein the mask window is used to align pixel points of the target image through geometric centers and collect areas corresponding to the pixel points; traversing the extended image in a predetermined manner according to the mask window to obtain the multiple areas collected multiple times by the mask window, wherein the position of the mask window corresponds one-to-one to the areas; and counting the pixel values ​​of the pixel points of the target image contained in each of the areas to determine a pixel histogram of the area.

[0008] Optionally, determining the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area includes: determining the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value respectively; determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; when the target area is a foreground area, determining the first segmentation threshold corresponding to the foreground area by using the maximum inter-class difference algorithm; when the target area is a background area, determining the second segmentation threshold corresponding to the background area by using the first grayscale value and the second grayscale value and the standard deviation of the pixel grayscale histogram.

[0009] Optionally, before determining the first grayscale value and the second grayscale value with the highest frequency in the pixel grayscale histogram, it also includes: fitting a mixed Gaussian model to the pixel grayscale histogram of the target area to determine the initial value of the mean and the initial value of the covariance of the pixel grayscale histogram; selecting an initial value of the mixing coefficient; calculating the initial value of the likelihood function and the posterior probability based on the mean, the covariance and the initial value of the mixing coefficient; determining the updated mean, covariance and mixing coefficient based on the posterior probability as a weighting coefficient; determining the updated value of the likelihood function based on the updated mean, covariance and mixing coefficient; when the updated value of the likelihood function meets the convergence condition, updating the pixel grayscale histogram according to the corresponding mean and covariance; when the updated value of the likelihood function does not meet the convergence condition, continuing to execute using the mean, covariance and mixing coefficient corresponding to the updated value as the initial value.

[0010] Optionally, determining whether the target area is a background area or a foreground area based on the first grayscale value and the second grayscale value includes: subtracting the first grayscale value from the second grayscale value and taking the absolute value to obtain the inter-class distance; when the inter-class distance is greater than or equal to a first preset multiple of the sum of standard deviations, determining that the target area is a foreground area; when the inter-class distance is less than a first preset multiple of the sum of standard deviations, determining that the target area is a background area; wherein the sum of standard deviations is the sum of a first standard deviation corresponding to the first grayscale value and a second standard deviation corresponding to the second grayscale value.

[0011] Optionally, when the target area is a foreground area, after determining the first segmentation threshold corresponding to the foreground area by the maximum inter-class difference algorithm, it also includes: determining the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels and the average grayscale of the background pixels; exponentially taking the ratio of the inter-class variance to the first compensation coefficient as the base value of a natural constant, and multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of light attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area.

[0012] Optionally, when the area is a background area, determining the second segmentation threshold corresponding to the background area through the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram includes: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is used as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is used as the minimum value; obtaining the second segmentation threshold by subtracting the standard deviation of a second preset multiple from the minimum value, wherein the second preset multiple is determined according to a normal distribution; and using the second segmentation threshold as the segmentation threshold for the background area.

[0013] Optionally, after traversing the multiple regions to determine the compensated regions corresponding to the multiple regions respectively, and combining all enhanced regions of the target image to generate a segmented image of the target image, the method further includes: inputting the segmented image of the target image into a trained segmentation model, and the segmentation model outputs the segmented regions of the majority of the segmented images.

[0014] According to another aspect of an embodiment of the present invention, an image segmentation device is also provided, including: an acquisition module, used to acquire a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of multiple areas of the target image, and the central pixel points of the multiple areas respectively correspond to the pixel points of the target image; a determination module, used to determine the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; a segmentation module, used to set the central pixel point to a first pixel value when the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold; and to set the central pixel point to a second pixel value when the pixel value of the central pixel point of the target area is less than the segmentation threshold, wherein the pixel difference between the first pixel value and the second pixel value is a preset threshold; and a traversal module, used to determine the first pixel value or the second pixel value corresponding to the multiple areas respectively by traversing the multiple areas, so as to generate a segmented image of the target image.

[0015] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned image segmentation methods

[0016] According to another aspect of an embodiment of the present invention, a computer storage medium is further provided, wherein the computer storage medium includes a stored program, wherein when the program is executed, the device where the computer storage medium is located is controlled to execute any one of the above-mentioned image segmentation methods.

[0017] In an embodiment of the present invention, a pixel grayscale histogram of a target area of ​​a target image is obtained, wherein the target area is one of a plurality of areas of the target image, and the central pixels of the plurality of areas respectively correspond to the pixels of the target image; a corresponding segmentation threshold of the target area is determined according to the pixel grayscale histogram of the target area; when the pixel value of the central pixel of the target area is greater than or equal to the segmentation threshold, the central pixel is set to a first pixel value; when the pixel value of the central pixel of the target area is less than the segmentation threshold, the central pixel is set to a second pixel value, and a compensated target area is obtained, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold. Range; by traversing multiple areas, determining the compensated areas corresponding to the multiple areas respectively, combining all the enhanced areas of the target image, generating a target image segmentation image, the pixel grayscale histogram of multiple areas of the target image, determining the segmentation threshold of the target area, and segmenting the target area, the target image is segmented according to the target area corresponding to the pixel points, so that the segmentation of each pixel point depends on the purpose of the pixels around it, thereby achieving the technical effect of improving the accuracy of target image segmentation, and further solving the technical problem in the related technology that the image segmentation accuracy is low due to the uneven illumination of the underwater segmented image and the blocking effect of the traditional segmentation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flow chart of an image segmentation method according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of an EM fitting grayscale histogram according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of segmentation results with and without compensation values ​​according to an embodiment of the present invention;

[0022] Figure 4 is a flowchart of image segmentation according to an embodiment of the present invention;

[0023] Figure 5is a schematic diagram of a grid-dividing result of an image by adaptive classification segmentation according to an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of an image segmentation device according to an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to an embodiment of the present invention, a method embodiment of an image segmentation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1 is a flow chart of an image segmentation method according to embodiment 1 of the present invention. Figure 1 As shown, the method comprises the following steps:

[0030] Step S102, obtaining a pixel grayscale histogram of a target area of ​​the target image, wherein the target area is one of multiple areas of the target image, and central pixel points of the multiple areas correspond to pixel points of the target image respectively;

[0031] Step S104, determining a corresponding segmentation threshold of the target area according to a pixel grayscale histogram of the target area;

[0032] Step S106, when the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold, the central pixel point is set to the first pixel value; when the pixel value of the central pixel point of the target area is less than the segmentation threshold, the central pixel point is set to the second pixel value, and a compensated target area is obtained, wherein the pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range;

[0033] Step S108, by traversing multiple regions, determining the compensated regions corresponding to the multiple regions respectively, combining all enhanced regions of the target image, and generating a segmented image of the target image.

[0034] Through the above steps, a pixel grayscale histogram of a target area of ​​a target image is obtained, wherein the target area is one of a plurality of areas of the target image, and the central pixels of the plurality of areas respectively correspond to the pixels of the target image; according to the pixel grayscale histogram of the target area, a corresponding segmentation threshold of the target area is determined; when the pixel value of the central pixel of the target area is greater than or equal to the segmentation threshold, the central pixel is set to a first pixel value; when the pixel value of the central pixel of the target area is less than the segmentation threshold, the central pixel is set to a second pixel value, and a compensated target area is obtained, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range. Surrounding; by traversing multiple areas, determining the compensated areas corresponding to the multiple areas respectively, combining all the enhanced areas of the target image, generating a target image segmentation image, a pixel grayscale histogram of multiple areas of the target image, determining the segmentation threshold of the target area, and segmenting the target area, the target image is segmented according to the target area corresponding to the pixel points, so that the segmentation of each pixel point depends on the pixel points around it, thereby achieving the technical effect of improving the accuracy of target image segmentation, and further solving the technical problem in the related art that the image segmentation accuracy is low due to the uneven illumination of the underwater segmented image and the blocking effect of the traditional segmentation method.

[0035] The above target image is an image with low contrast, specifically, an image with a contrast less than a preset contrast. The low contrast is caused by uneven light distribution, which is mainly caused by fill light. Since the fill light LEDs are concentrated around the camera, the captured images basically have a bright center, and the light decays from the center to the surroundings. The grayscale value at the edges is too low, resulting in the feed particles at the edges being extremely unclear, which brings great difficulties to image processing.

[0036] The above-mentioned target area is a plurality of areas centered on all the pixels of the target image, and the size and shape of the area are preset, and the preset size of the rectangle can be selected as the size parameter of the above-mentioned area. In step S102, the pixel point at the center of the area to be acquired can be first determined, and all the pixels included in the area range can be determined according to the preset size parameter of the area, and then the grayscale of the pixel point is acquired to generate the histogram of the area, and the grayscale histogram of all the areas is acquired by traversing all the pixels of the target image. The above-mentioned target area is any one of the above-mentioned multiple areas, that is, when analyzing, the analysis and segmentation are performed in units of areas.

[0037] It should be noted that when the pixel point is the pixel point at the edge of the image of the target image, there is a situation where the above-mentioned area exceeds the edge of the image. Therefore, before dividing the target image into regions, the target image can be offset and expanded according to the size of the above-mentioned area. The expanded area beyond the target image can be replaced by a unified maximum grayscale or minimum grayscale to meet the needs of regional histogram analysis.

[0038] Optionally, obtaining a pixel grayscale histogram of a target area of ​​a target image includes: subtracting one from half of a preset size of a mask window to obtain an extended size for expanding the target image, wherein the mask window is square; expanding the target image according to the extended size to obtain an extended image after expansion, wherein the mask window is used to align pixel points of the target image through geometric centers and collect areas corresponding to the pixel points; traversing the extended image in a predetermined manner according to the mask window to obtain multiple areas collected by the mask window multiple times, wherein the position of the mask window corresponds one-to-one to the area; and counting the pixel values ​​of the pixel points of the target image contained in each area to determine a pixel histogram of the area.

[0039] The size of the mask window can be the same as the size parameter of the region, and is used to obtain the grayscale value of the pixel corresponding to the region by means of the mask window. The predetermined method can be to traverse all the pixel points of the target image according to the preset path, for example, starting from the upper left corner, traversing row by row from left to right, that is, first traversing from the pixel point at the upper left corner of the target image, that is, the left end pixel point of the first row, to the right end pixel point of the first row one by one, and then traversing from the left end pixel point to the right end pixel point of the next row. Obtain multiple regions acquired by the mask window multiple times, and determine the pixel histogram according to the pixel values ​​of the pixels in the region. Each time the pixel points corresponding to the region are acquired through the mask window, the traversed pixel points are aligned with the geometric center of the mask window, so that it can be determined that when the mask window is used to acquire the pixel points of the region at the pixel points at the edge of the target image, it is necessary to ensure that there are filled pixels within a distance of half of the preset size minus one outside the target image region, otherwise due to the image size, the mask window cannot acquire the pixel points of the region in the above manner.

[0040] The target image is expanded according to the expansion size to obtain the expanded image, and the target object can be expanded to a range equal to or larger than the expansion size to ensure that the mask window captures the target image.

[0041] According to the above pixel grayscale histogram of the target area, the corresponding segmentation threshold of the target area can be determined by selecting one or more grayscale values ​​with higher pixel frequency through the pixel grayscale histogram. In theory, the grayscale value with more than half of the pixel frequency can be used as the segmentation threshold. However, in order to take into account the complex grayscale changes, the accuracy of determining the segmentation threshold by simple pixel frequency is low. This embodiment selects the first two grayscale values ​​with the highest pixel frequency to divide the target area into an important foreground area or an unimportant background area, and adopts different segmentation thresholds for different foreground areas and background areas, thereby ensuring segmentation accuracy and improving data processing efficiency.

[0042] Optionally, determining the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area includes: determining the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value respectively; determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; when the target area is a foreground area, determining the first segmentation threshold corresponding to the foreground area by using the maximum inter-class difference algorithm; when the target area is a background area, determining the second segmentation threshold corresponding to the background area by using the first grayscale value and the second grayscale value and the standard deviation of the pixel grayscale histogram.

[0043] When determining the first grayscale value and the second grayscale value, the maximum inter-class difference method can be used. Specifically, t is the segmentation threshold of the foreground and the background. The proportion of the number of foreground pixels in the entire image is ω0, and the average grayscale of the foreground part is μ0, which is the first grayscale value; the proportion of the number of background pixels in the image is ω1, and the average grayscale of the background is μ1, which is the second grayscale value. Then the total average grayscale of the image is:

[0044] μ=ω0×μ0+ω1×μ1. (1)

[0045] Traverse t from the minimum grayscale value to the maximum grayscale value, when there is t such that the value

[0046] g=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (2)

[0047] When t is the maximum, it is the optimal threshold for segmentation.

[0048] μ=ω0×μ0+ω1×μ1, (3)

[0049] ω0+ω1=1. (4)

[0050] Substituting equation (3) and equation (4) into equation (2), we can simplify and obtain

[0051] g=ω0×ω1×(μ0-μ1) 2 . (5)

[0052] g is the criterion for judging the quality of the threshold. The larger the value, the better the threshold is selected, and the better the foreground and background can be separated. The value range of g is 0-1.

[0053] Otsu can further understand it from a mathematical perspective. Formula (5) is actually the formula for calculating the inter-class variance of foreground and background. In mathematical terms, variance is a measure of the degree of data dispersion. The goal of image segmentation is to divide the image into the most appropriate foreground and background categories. The g value calculated by formula (5) is the variance that measures the uniformity of grayscale distribution. The larger the value, the greater the difference between the two classes that make up the image. When some foreground pixels are mistakenly classified as background pixels or background pixels are mistakenly classified as foreground pixels, the difference between the foreground and background categories will become smaller, and formula (5) will not take the maximum value at this time. Therefore, if there is a segmentation threshold that maximizes the inter-class variance, it means that the foreground and background categories are most separable at this time.

[0054] Optionally, determining whether the target area is a background area or a foreground area based on the first grayscale value and the second grayscale value includes: subtracting the first grayscale value from the second grayscale value and taking the absolute value to obtain the inter-class distance; when the inter-class distance is greater than or equal to a first preset multiple of the sum of standard deviations, determining the target area as a foreground area; when the inter-class distance is less than a first preset multiple of the sum of standard deviations, determining the target area as a background area; wherein the sum of standard deviations is the sum of a first standard deviation corresponding to the first grayscale value and a second standard deviation corresponding to the second grayscale value.

[0055] After the first grayscale value and the second grayscale value are determined, the region can be divided according to the corresponding segmentation threshold to determine the foreground part and the background part. Specifically, by fitting the grayscale histogram through the EM algorithm, the double peak of the grayscale histogram can be determined, and the distance between the two double peaks after fitting is used as the basis for judging whether the mask window is the foreground window or the background window. The discriminant expression is as follows:

[0056]

[0057] This expression means that the difference between the means of the two Gaussian distributions after fitting is used as the inter-class distance. If the distance is greater than or equal to 1.3 times the sum of the standard deviations of the two distributions, the window is considered to be a foreground window; conversely, if the distance is less than 1.3 times the sum of the standard deviations of the two distributions, the window is considered to be a background window.

[0058] That is, the inter-class distance is determined according to the first grayscale value and the second grayscale value, and the difference between the means of the first grayscale value and the second grayscale value may be determined. The preset multiple may be 1.3.

[0059] Optionally, before determining the first grayscale value and the second grayscale value with the highest frequency in the pixel grayscale histogram, it also includes: fitting a mixed Gaussian model to the pixel grayscale histogram of the target area to determine the initial value of the mean and the initial value of the covariance of the pixel grayscale histogram; selecting an initial value of the mixing coefficient; calculating the initial value of the likelihood function and the posterior probability according to the initial values ​​of the mean, covariance and mixing coefficient; determining the updated mean, covariance and mixing coefficient according to the posterior probability as a weighting coefficient; determining the updated value of the likelihood function according to the updated mean, covariance and mixing coefficient; when the updated value of the likelihood function meets the convergence condition, updating the pixel grayscale histogram according to the corresponding mean and covariance; when the updated value of the likelihood function does not meet the convergence condition, continuing to execute using the mean, covariance and mixing coefficient corresponding to the updated value as the initial value.

[0060] Mixed Gaussian model fitting, also known as mixed Gaussian distribution, is a linear combination of multiple single Gaussian distributions with different coefficients. By adjusting the mean, covariance and coefficients of each Gaussian distribution, any continuous distribution can be fitted to any accuracy.

[0061] Consider a combination of K Gaussian distributions:

[0062]

[0063] This is a mixture of Gaussian distributions. Each Gaussian distribution N(x|μ k ,∑ k ) is called the mixture component. Each component has its own mean μ k , which is the initial value of the mean mentioned above, and the covariance ∑ k , which is the initial value of the covariance mentioned above. Parameter π k is called the mixing coefficient and it satisfies:

[0064]

[0065] Formula (6) can be written in the form of marginal distribution:

[0066]

[0067] Comparing equation (6) with equation (8), we can see that π k =p(k). This formula can be regarded as the prior probability of the kth distribution in the mixed distribution. And Ν(x|μ k ,∑ k )=p(x|k) is the conditional probability of x based on k. The form of the mixed Gaussian distribution is controlled by constants π, μ and ∑. Here we use π={π1,…,π k},μ={μ1,…,μ k} and ∑={∑1,…,∑ k}. One way to estimate these constants is through maximum likelihood estimation. First, take the logarithm of the maximum likelihood function:

[0068]

[0069] Here x={x1,…,x k}. Since the maximum likelihood function contains K consecutive additions of single terms, it is difficult to obtain a closed analytical solution for the equation after the derivative of this function. Therefore, one way to maximize this equation is to use iterative numerical optimization methods. Another way is to apply the framework of the EM algorithm to solve it.

[0070] The EM algorithm is applicable to the case of data sets containing latent variables. For the mixed Gaussian distribution of equation (6), we introduce a K-dimensional binary vector variable z. This variable contains K components, among which only one component z k is 1, and the other components are 0. Therefore, each component z k The value of z satisfies k ∈{0,1} and∑k z k =1. Then, the vector z has a total of K possible states. The marginal distribution of z can be expressed by the mixing coefficient π k express:

[0071] p(z k =1) = π k . (10)

[0072] From formula (7), we can write:

[0073]

[0074] Similarly, the conditional probability of x under z is a Gaussian distribution:

[0075] p(x|z k =1) = Ν(x|μ k ,Σ k ) (12)

[0076] This formula can also be written as:

[0077]

[0078] The joint distribution is given by p(z)p(x|z), then the marginal distribution of x can be obtained by adding all possible states of z

[0079]

[0080] The above formula is derived from formulas (11) and (13). Therefore, the marginal distribution of x is a mixture of Gaussian distributions like formula (6). If there are observations x1,…,x N , because the marginal distribution is already expressed as p(x) = Σ z p(x,z), then for each observation data point x n , there is an implicit variable z n . Then, for the derivation of mixed Gaussian distributions containing latent variables, it is much easier to use the joint distribution p(x,z) than the marginal distribution p(x).

[0081] For the likelihood function of formula (9), k Taking the derivative and setting it equal to zero, we get:

[0082]

[0083] remember

[0084]

[0085] Multiply both sides of equation (15) by (Assuming that the covariance matrix is ​​non-singular), and then simplifying, we can get

[0086]

[0087] definition:

[0088]

[0089] Here N k It can be regarded as the number of valid data points corresponding to k classes. The mean μ of the kth Gaussian distribution is k It is calculated by the weighted average of all data points in the data set. n The weight factor is the posterior probability γ(z nk ) is given.

[0090] If the likelihood function is Σ k The derivative of is zero, then we can get

[0091]

[0092] From formula (19), we can see that the posterior probability γ(z nk ) as the weighting coefficient, and the number of effective points is in the denominator. Then make the likelihood function π k The derivative of is zero. Due to the constraints of formula (7), the Lagrange multiplier method is applied here to obtain the Lagrange function

[0093]

[0094] π k Find the partial derivative and combine it with the constraints to get

[0095]

[0096] Once again, the posterior probability γ(z nk ) appears in the expression. Multiply both sides by π k , and substituting the constraints and λ=-N, we can get

[0097]

[0098] Although equations (17), (19), and (22) are the coefficients μ of the mixed Gaussian distribution k ,Σ k and π k The solution is in the form of, but these three equations cannot form a closed solution. Because according to equation (16), the posterior probability γ(z nk ) depends on these three parameters, and the solutions of the three parameters all depend on γ(z nk). However, the inspiration brought by this result is that the solution to the parameters of the mixed Gaussian distribution depends on a simple iterative solution process of the maximum likelihood problem. This is also a special case of the EM algorithm applied to the Gaussian mixture distribution. Therefore, for the solution of the parameters of the mixed Gaussian distribution, first give the mean μ k , the covariance matrix Σ k and the mixing coefficient π k Select the initial value, and then iterate in the two steps of parameter estimation until the accuracy requirements of the parameters are met. This is the basic step of the EM algorithm.

[0099] In the EM algorithm, the two steps of iteration are called the E (expectation) step and the M (maximization) step. In the E step, the current value of the coefficient is used to calculate the value of the posterior probability, that is, formula (16); the posterior probability γ(z nk ), in the M step, the coefficient μ is calculated using the posterior probability according to equations (17), (19) and (22) respectively. k ,Σ k and π k In fact, each iteration of the E step and the M step can ensure that the log likelihood function increases, so the method is convergent. However, since the EM algorithm is a greedy algorithm, the likelihood function may have multiple local optimal values, so the EM algorithm cannot guarantee convergence to the global maximum value.

[0100] Here is a summary of the steps of the EM algorithm. Given a mixed Gaussian model with hidden variables, if you want to maximize its likelihood function, then

[0101] 1. Given the mean μ k , covariance Σ k and the mixing coefficient π k , and calculate the initial value of the log likelihood function.

[0102] 2. Step E. Calculate the posterior probability γ(z nk ).

[0103] 3.M step. Calculate μ in turn by using equations (17), (19) and (22) and the value of the posterior probability k ,Σ k and π k The updated value of .

[0104] 4. Calculate the value of the log likelihood function (9) and check the convergence of the log likelihood function or coefficient. If the convergence condition is not met, return to step 2 to continue.

[0105] Optionally, in the case where the target area is the foreground area, after determining the first segmentation threshold corresponding to the foreground area by the maximum inter-class difference algorithm, it also includes: determining the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels and the average grayscale of the background pixels; exponentially taking the ratio of the inter-class variance to the first compensation coefficient as the base value of a natural constant, multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of light attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area.

[0106] Optionally, in the case where the area is a background area, determining the second segmentation threshold corresponding to the background area through the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram includes: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is taken as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is taken as the minimum value; obtaining the second segmentation threshold by subtracting the standard deviation of the second preset multiple from the minimum value, wherein the second preset multiple is determined according to the normal distribution; and using the second segmentation threshold as the segmentation threshold for the background area.

[0107] After judging the window, Otsu can be applied conditionally. If the judgment result is the foreground window, Otsu is applied in the window, and the threshold value Th corresponding to the center pixel of the window is bimodal =Th Otsu Otherwise, if the judgment result is the background window, the threshold corresponding to the central pixel is the smaller μ value minus 2.5 times the corresponding standard deviation. The specific expression is as follows:

[0108]

[0109] After obtaining the corresponding threshold, the gray value of the central pixel is compared with the threshold. If the gray value of the central pixel is greater than the threshold, the corresponding pixel value on the binary image is set to 0, that is, black; conversely, if the gray value of the central pixel is less than the threshold, the corresponding pixel value on the binary image is set to 255, that is, white.

[0110] Regarding the method of obtaining the threshold value of the background image here, the present invention applies the knowledge of Gaussian distribution probability statistics. Gaussian distribution is also called normal distribution. For the standard normal distribution, the probability that the independent variable falls within the interval of the mean plus or minus 2.5 times the standard deviation is 0.9938. Therefore, theoretically speaking, when calculating the threshold value above, the threshold value obtained by subtracting the corresponding 2.5 times the standard deviation from the smaller μ is smaller than the grayscale value of 99.38% of the pixels in the mask window. It is natural to use this sufficiently small threshold as the threshold value of the background window.

[0111] Optionally, after traversing multiple regions to determine the compensated regions corresponding to the multiple regions respectively and combining all enhanced regions of the target image to generate a segmented image of the target image, the method further includes: inputting the segmented image of the target image into a trained segmentation model, and the segmentation model outputs the segmented regions of the majority of the segmented images.

[0112] It should be noted that this embodiment also provides an optional implementation, which is described in detail below.

[0113] This embodiment combines the illumination distribution characteristics of underwater images, and on the basis of local threshold segmentation, uses the gray value information of the pixel neighborhood to determine the corresponding threshold, successfully solving the problem of block effect. In terms of the method of determining the threshold in the neighborhood, this embodiment adopts Otsu, i.e., the maximum inter-class variance method. However, Otsu only has a good segmentation effect on images with a double-peaked gray distribution histogram (i.e., images with obvious differences between the foreground and the background), so before calculating the threshold, the EM algorithm is used to fit the gray distribution histogram to determine the inter-class distance between the background and the foreground. The inter-class distance can be used to determine whether the neighborhood has a double-peaked characteristic of the background and the foreground coexisting, thereby determining the application conditions of Otsu. Finally, because the proportion of the background category is much greater than that of the foreground category, the threshold obtained by Otsu will have a shift to the category with a large proportion, and the shift depends on the size of the proportion of the large category. Therefore, this embodiment starts from the inter-class variance obtained by Otsu, uses the inter-class variance as a measure of the proportion of the two classes, and derives a threshold compensation function with the inter-class variance as an independent variable. Finally, each pixel is compared with its corresponding threshold to complete the segmentation. It is very robust to different underwater lighting conditions and turbidity.

[0114] The technical features involved in this implementation are described in detail below:

[0115] 1. Otsu and its segmentation principle: The Otsu algorithm (Otsu method) is also called the maximum inter-class difference method. This algorithm is generally considered to be the best algorithm for threshold selection in image segmentation because of its simple calculation, adaptability, and is not affected by image brightness and contrast. It is widely used in the field of digital image processing. The application of Otsu is inseparable from the analysis of the grayscale distribution histogram of digital images. Only one-dimensional grayscale histogram is used in Otsu. The basic idea is: get the grayscale image of the image and divide it into two parts: background and foreground. Let t be the segmentation threshold of the foreground and background, the proportion of foreground pixels in the entire image is ω0, and the average grayscale of the foreground part is μ0; the proportion of background pixels in the image is ω1, and the average grayscale of the background is μ1. Then the total average grayscale of the image is:

[0116] μ=ω0×μ0+ω1×μ1. (1)

[0117] Traverse t from the minimum grayscale value to the maximum grayscale value, when there is t such that the value:

[0118] g=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (2)

[0119] When t is the maximum, it is the optimal threshold for segmentation.

[0120] μ=ω0×μ0+ω1×μ1, (3)

[0121] ω0+ω1=1. (4)

[0122] Substituting equation (3) and equation (4) into equation (2), we can simplify and obtain:

[0123] g=ω0×ω1×(μ0-μ1) 2 . (5)

[0124] g is the criterion for judging the quality of the threshold. The larger the value, the better the threshold is selected, and the better the foreground and background can be separated. The value range of g is 0-1.

[0125] Otsu can further understand it from a mathematical perspective. Formula (5) is actually the formula for calculating the inter-class variance of foreground and background. In mathematical terms, variance is a measure of the degree of data dispersion. The goal of image segmentation is to divide the image into the most appropriate foreground and background categories. The g value calculated by formula (5) is the variance that measures the uniformity of grayscale distribution. The larger the value, the greater the difference between the two classes that make up the image. When some foreground pixels are mistakenly classified as background pixels or background pixels are mistakenly classified as foreground pixels, the difference between the foreground and background categories will become smaller, and formula (5) will not take the maximum value at this time. Therefore, if there is a threshold that maximizes the inter-class variance, it means that the foreground and background categories are most separable at this time.

[0126] The maximum inter-class difference method is generally used in the case where the grayscale histogram is bimodal. [7] gives the application of this method in the case of trimodal, but it needs some extension. For the case of bimodal, the maximum inter-class difference method is always stable and convergent, which is manifested as the image of g is smooth and unimodal. However, the proof of this part is not given in [7].

[0127] Otsu is very sensitive to noise and target size, and it only produces good segmentation results for images with a double-peaked grayscale histogram. When the size of the foreground and background is very different, the inter-class variance criterion function may be double-peaked or multi-peaked, and the Otsu segmentation effect is not good at this time. But in general, the maximum inter-class variance method is the fastest and most efficient global threshold segmentation method.

[0128] 2. Principle of Mixed Gaussian Distribution and EM Algorithm: EM algorithm (Expectation Maximization Algorithm) is one of the ten classic algorithms in the field of pattern recognition and machine learning. It was proposed by Dempster in 1977. EM algorithm is an iterative algorithm used for maximum likelihood estimation or maximum posterior probability estimation of probability parameter models containing latent variables. EM algorithm is often used in data clustering analysis. In probability statistics, Gaussian distribution is the most important type of distribution, and it has many important analytical properties. However, when applied to actual data analysis, single Gaussian distribution also has many limitations and defects. In view of these shortcomings, the concept of mixture distributions was proposed. Mixture distribution is a data probability model composed of many basic types of distributions (such as Poisson distribution, Gaussian distribution, etc.) linearly combined. Mixed Gaussian distribution is a linear combination of multiple single Gaussian distributions with different coefficients. By adjusting the mean, covariance and coefficients of each Gaussian distribution, any continuous distribution can be fitted to any precision.

[0129] Consider a combination of K Gaussian distributions:

[0130]

[0131] This is a mixture of Gaussian distributions. Each Gaussian distribution N(x|μ k ,∑ k ) is called the mixture component. Each component has its own mean μ k and covariance∑ k Parameter π k is called the mixing coefficient and it satisfies:

[0132]

[0133] Formula (6) can be written in the form of marginal distribution:

[0134]

[0135] Comparing equation (6) with equation (8), we can see that π k =p(k). This formula can be regarded as the prior probability of the kth distribution in the mixed distribution. And Ν(x|μ k ,∑ k )=p(x|k) is the conditional probability of x based on k. The form of the mixed Gaussian distribution is controlled by the constants π, μ and ∑. Here we use π={π1,…,π k},μ={μ1,…,μ k} and ∑={∑1,…,∑ k}. One way to estimate these constants is through maximum likelihood estimation. First, take the logarithm of the maximum likelihood function:

[0136]

[0137] Here x={x1,…,x k}. Since the maximum likelihood function contains K consecutive additions of single terms, it is difficult to obtain a closed analytical solution for the equation after the derivative of this function. Therefore, one way to maximize this equation is to use iterative numerical optimization methods. Another way is to apply the framework of the EM algorithm to solve it.

[0138] As mentioned above, the EM algorithm is applicable to the case of a data set containing latent variables. For the mixed Gaussian distribution of equation (6), we introduce a K-dimensional binary vector variable z. This variable contains K components, among which only one component z k is 1, and the other components are 0. Therefore, each component z k The value of z satisfies k ∈{0,1} and∑ k z k =1. Then, the vector z has a total of K possible states. The marginal distribution of z can be expressed by the mixing coefficient π k express:

[0139] p(z k =1) = π k . (10)

[0140] From formula (7), we can write:

[0141]

[0142] Similarly, the conditional probability of x under z is a Gaussian distribution:

[0143] p(x|z k =1) = Ν(x|μ k ,Σ k ) (12)

[0144] This formula can also be written as:

[0145]

[0146] The joint distribution is given by p(z)p(x|z), then the marginal distribution of x can be obtained by adding all possible states of z

[0147]

[0148] The above formula is derived from formulas (11) and (13). Therefore, the marginal distribution of x is a mixture of Gaussian distributions like formula (6). If there are observations x1,…,x N , because the marginal distribution is already expressed as p(x) = Σ z p(x,z), then for each observation data point x n , there is an implicit variable z n . Then, for the derivation of mixed Gaussian distributions containing latent variables, it is much easier to use the joint distribution p(x,z) than the marginal distribution p(x).

[0149] For the likelihood function of formula (9), k Taking the derivative and setting it equal to zero, we get:

[0150]

[0151] remember:

[0152]

[0153] Multiply both sides of equation (15) by (Assuming that the covariance matrix is ​​non-singular), and then simplifying, we can get:

[0154]

[0155] definition:

[0156]

[0157] Here N k It can be regarded as the number of valid data points corresponding to k classes. The mean μ of the kth Gaussian distribution is k It is calculated by the weighted average of all data points in the data set. n The weight factor is the posterior probability γ(z nk ) is given.

[0158] If the likelihood function is Σ k The derivative of is zero, then we can get:

[0159]

[0160] From formula (19), we can see that the posterior probability γ(z nk ) as the weighting coefficient, and the number of effective points is in the denominator. Then make the likelihood function π k The derivative of is zero. Due to the constraints of formula (7), the Lagrange multiplier method is applied here to obtain the Lagrange function:

[0161]

[0162] π k Find the partial derivative and combine it with the constraints to get:

[0163]

[0164] Once again, the posterior probability γ(z nk ) appears in the expression. Multiply both sides by π k , and substituting the constraints and λ=-N, we can get:

[0165]

[0166] Although equations (17), (19), and (22) are the coefficients μ of the mixed Gaussian distribution k ,Σ k and π k The solution is in the form of, but these three equations cannot form a closed solution. Because according to equation (16), the posterior probability γ(z nk ) depends on these three parameters, and the solutions of the three parameters all depend on γ(z nk ). However, the inspiration brought by this result is that the solution to the parameters of the mixed Gaussian distribution depends on a simple iterative solution process of the maximum likelihood problem. This is also a special case of the EM algorithm applied to the Gaussian mixture distribution. Therefore, for the solution of the parameters of the mixed Gaussian distribution, first give the mean μ k , the covariance matrix Σ k and the mixing coefficient π k Select the initial value, and then iterate in the two steps of parameter estimation until the accuracy requirements of the parameters are met. This is the basic step of the EM algorithm.

[0167] In the EM algorithm, the two steps of iteration are called the E (expectation) step and the M (maximization) step. In the E step, the current value of the coefficient is used to calculate the value of the posterior probability, that is, formula (16); the posterior probability γ(z nk ), in the M step, we use the posterior probability value to calculate the coefficient μ according to equations (17), (19) and (22) respectively. k ,Σ k and π k In fact, each iteration of the E step and the M step can ensure that the log likelihood function increases, so the method is convergent. However, since the EM algorithm is a greedy algorithm, the likelihood function may have multiple local optimal values, so the EM algorithm cannot guarantee convergence to the global maximum value.

[0168] Here is a summary of the steps of the EM algorithm. Given a mixed Gaussian model with hidden variables, if you want to maximize its likelihood function, then:

[0169] 5. Given μ k ,Σ k and π k , and calculate the initial value of the log likelihood function.

[0170] 6. Step E. Calculate the posterior probability γ(z nk ).

[0171] 7. Step M. Calculate μ in turn by using equations (17), (19) and (22) and the value of the posterior probability k ,Σ k and π k The updated value of .

[0172] 8. Calculate the value of the log likelihood function (9) and check the convergence of the log likelihood function or coefficient. If the convergence condition is not met, return to step 2 to continue.

[0173] 3. Adaptive classification segmentation: This embodiment proposes an adaptive classification segmentation. In this algorithm, the concept of a mask window is introduced. Specifically, a mask window is a small movable window. For an image with a mask window, only the pixels within the mask window will be selected for processing. During the processing, the mask window starts from the upper left corner pixel of the image and scans the entire image row by row and pixel by pixel. In this way, after each mask window moves, the grayscale information of the neighborhood pixels inside the window is analyzed to obtain the threshold value within each window. But one thing to note here is that the threshold value obtained for each window is only relative to the center pixel of the window. Therefore, each pixel has a threshold value obtained from the grayscale information of its neighborhood. By comparing each pixel with the obtained threshold value, the corresponding pixel result of the binary image can be obtained.

[0174] Therefore, before processing each image, the border of the image must be expanded. Otherwise, the border pixels cannot be processed correctly. For example, if the size of the mask window is 49*49, the four borders of the image to be processed must be expanded by (49-1) / 2=24 lengths. When the mask window is scanned from the upper left corner, the center pixel happens to start from the upper left corner pixel of the original image. Then process line by line and set the corresponding threshold for each pixel of the original image.

[0175] Regarding the segmentation method within the mask window, this embodiment considers using Otsu. However, as mentioned above, Otsu only has a good segmentation effect on images with a double-peaked grayscale distribution histogram. Therefore, Otsu is not applicable to all mask windows.

[0176] Since the grayscale distribution histogram of the mask window containing the foreground object is basically bimodal, while the grayscale distribution histogram of the pure background mask window is unimodal, Otsu is not applicable to the background mask window, and applying Otsu to the background mask window will cause the threshold to deviate.

[0177] In order to solve this problem, this embodiment proposes to use the EM algorithm to determine the grayscale distribution of each small window, that is, before processing the pixels in the mask window each time, the EM algorithm is used to fit the grayscale distribution histogram of the mask window with a mixed Gaussian model. represents n pixels within the mask window. Assume can be fitted by a mixed Gaussian model, then represents the latent variable of the mixed model, and the latent variables of different grayscale data points are different. Therefore, z i =j represents x i is generated by the jth distribution of the mixture model. Here we only care whether the pixels in the mask window can be divided into two categories, so K is set to 2. represent foreground pixels and background pixels respectively. Assume that the pixels of each class obey Gaussian distribution:

[0178] p(x i |z i =j,θ)=Ν(x i |μ j ,σ j ) (twenty three)

[0179] μ j and σ j Represent the mean and standard deviation of the jth Gaussian distribution. θ=[ω T ,μ T ,σ T ] T =[ω1,…,ω K ,μ1,…,μ K ,σ1,…,σ K ] is the coefficient vector of the mixed model. Then the mixed Gaussian model can be written as:

[0180] p(x i |θ)=Σ j p(x i |z i =j,θ)·p(z i =j|θ) (24)

[0181] Weight coefficient ω j =p(z i =j|θ) is the prior probability of the jth distribution. Then we can get:

[0182]

[0183] The EM algorithm recursively obtains the maximum likelihood estimate of θ. The log likelihood function can be written as:

[0184]

[0185] Assume that at iteration t, θ (t-1) is the estimated value of θ at the t-1th iteration. Then θ (t) is the value of θ to be estimated now. In step E, the posterior probability is first calculated using the Bayesian formula:

[0186]

[0187] In the M-step, θ is calculated by maximizing (t) :

[0188]

[0189] After simplification and derivation, the parameter estimation formula of the mixed Gaussian model can be obtained:

[0190]

[0191]

[0192]

[0193] For each mask window, iteratively calculate equations (29)-(31) until the coefficients converge. Convergence is defined as:

[0194]

[0195] The value is less than 1×10 -3 The EM algorithm is very sensitive to the selection of initial values. A suitable initial value can reduce the number of iterations and thus converge to the extreme value more quickly. In this embodiment, the two gray values ​​with the highest frequency in the gray histogram are selected as and After experiments, the initial value of the standard deviation is set between 2.0 and 9.0. In multiple fitting experiments, the EM algorithm sometimes converges to a local optimal value and fails to fit the grayscale distribution histogram. To avoid this, the program checks ω after each fitting. j If |ω1-ω2| is greater than 0.98, the program will reset the initial values ​​of each parameter and rerun the EM algorithm.

[0196] Figure 2 is a schematic diagram of an EM fitting grayscale histogram according to an embodiment of the present invention, such as Figure 2As shown in the figure, the results of fitting the grayscale histogram using the EM algorithm are given, including the fitting results of the foreground window and the background window under different lighting conditions. The left side (a) is a typical bimodal grayscale histogram of the foreground window. It can be seen from the figure that it is well fitted by two Gaussian distributions. The right side (b) is the fitting result of the background window. The grayscale histogram is basically unimodal. It can be seen from the figure that the two Gaussian distributions are very close after EM fitting. Therefore, the distance between the two Gaussian distributions after fitting is used as the basis for judging whether the mask window is a foreground window or a background window. The discriminant expression is as follows:

[0197]

[0198] This expression means that the difference between the means of the two Gaussian distributions after fitting is used as the inter-class distance. If the distance is greater than 1.3 times the sum of the standard deviations of the two distributions, the window is considered to be a foreground window; conversely, if the distance is less than 1.3 times the sum of the standard deviations of the two distributions, the window is considered to be a background window. After judging the window, Otsu can be applied conditionally. If the judgment result is a foreground window, Otsu is applied in the window, and the threshold Th corresponding to the center pixel of the window is bimodal =Th Otsu Otherwise, if the judgment result is the background window, the threshold corresponding to the central pixel is the smaller μ value minus 2.5 times the corresponding standard deviation. The specific expression is as follows:

[0199]

[0200] After obtaining the corresponding threshold, the gray value of the central pixel is compared with the threshold. If the gray value of the central pixel is greater than the threshold, the corresponding pixel value on the binary image is set to 0, that is, black; conversely, if the gray value of the central pixel is less than the threshold, the corresponding pixel value on the binary image is set to 255, that is, white.

[0201] Regarding the method of obtaining the threshold value of the background image here, the present invention applies the knowledge of Gaussian distribution probability statistics. Gaussian distribution is also called normal distribution. For the standard normal distribution, the probability that the independent variable falls within the interval of the mean plus or minus 2.5 times the standard deviation is 0.9938. Therefore, theoretically speaking, when calculating the threshold value above, the threshold value obtained by subtracting the corresponding 2.5 times the standard deviation from the smaller μ is smaller than the grayscale value of 99.38% of the pixels in the mask window. It is natural to use this sufficiently small threshold as the threshold value of the background window.

[0202] 4. Inter-class variance and compensation function: For a double-peaked mask window, the most ideal situation is that the threshold value obtained falls exactly between the double peaks of the grayscale histogram, which can perfectly separate the two peaks. However, in the actual application of this embodiment, the threshold value obtained by Otsu usually deviates from the lowest valley between the two peaks. For a grayscale distribution histogram with two peaks with greatly different proportions, this is mainly because when calculating the inter-class variance, the large-proportion class with more data points has a larger number of samples, so the weight contributed when calculating the variance is larger, thus pulling the threshold closer to itself. In the mask window of the foreground object, the number of pixels occupied by the target accounts for only a small proportion of all pixels. This can be seen from Figure 2 Therefore, this pulling-in effect is quite obvious, and the thresholds obtained by Otsu are larger than the optimal threshold.

[0203] To solve this problem, a compensation value δ is added to the threshold obtained by Otsu. The specific derivation is as follows. According to Otsu, the calculation formula of the between-class variance is

[0204]

[0205] in They represent the proportion of foreground pixels in the entire image, the proportion of background pixels in the image, the average grayscale of the foreground part, and the average grayscale of the background part. Formula (35) reflects the balance between the two classes and the distance between classes. If the proportion of the two classes is relatively even and the segmentation is relatively good, then Larger, on the contrary, It is easy to see from formula (35) that when hour, The maximum value reaches 0.25. At this time, the compensation value does not need to be very large, and it can even be zero. When the balance between the two categories is small and the variance between the categories is small, the value of the compensation amount δ should be larger to pull the threshold closer to the category with a small proportion, making the threshold more accurate. Here, a formula similar to the Gaussian distribution formula is used to characterize the value of this compensation value:

[0206]

[0207] In this formula, the constant c can be regarded as the weighted average inter-class variance from one class center to another class center. Finally, the exponential function of the above formula is multiplied by a coefficient a. This coefficient is used to adjust the value range of the compensation value. After adding the compensation value, the threshold of the bimodal window is given by the following formula:

[0208]

[0209] The calculation of the compensation value is adaptive because the coefficients a and c are determined based on the separability and proportion between the two classes. After experimental determination, the value ranges of coefficient a and coefficient c are (4, 15) and (3, 30) respectively. The segmentation effect after applying the adaptive compensation value is shown in the figure below.

[0210] Figure 3 is a schematic diagram of segmentation results with and without compensation values ​​according to an embodiment of the present invention, such as Figure 3 As shown in Figure 1, (a) is the original mask window. (d) is its grayscale distribution histogram. Figure 3 It can be seen that the proportions of the two classes are unequal. The vertical line on the right indicates the threshold obtained by Otsu. Figure 3 (d) shows that it is around 130. The vertical line on the left is the threshold value obtained by subtracting the compensation value from Otsu, which is around 126. (b) is the result of segmentation using the Otsu threshold, and (c) is the result of segmentation using Otsu minus the compensation value. From these two images (b) and (c), it can be seen that (c) has less noise than (b).

[0211] In the calculation formula of the compensation value δ, the role of coefficient a is to adjust the range of the compensation value. The role of coefficient c is to adjust the rate of change of the δ value. Since the turbidity of water in different shots is different during the underwater image shooting process, and the distance between the camera and the target is also different, the coefficients of the compensation value δ calculation formula for each image obtained from the experiment are different. In order to derive the formula for calculating coefficients a and c, first consider an extreme case: the case where the water turbidity is ∞. In this case, all foreground objects cannot be seen, and any pixels in the mask window can only be classified as background. At this time, the inter-class variance is 0, and the compensation value δ is α.

[0212] Another extreme case is when the entire underwater image is filled with foreground objects. In this case, since the entire image is filled with foreground objects, there is no distinction between the background and the foreground, and the segmentation algorithm cannot identify and classify them. This situation does not need to be considered in actual applications. In addition, the mask window will not be filled with foreground objects in the application. Because the size of the mask window must be larger than the largest foreground object when selected, so that segmentation can be performed effectively. The last extreme case is that the water is clear, the transmittance is good, the refraction and scattering rates are small, and the foreground objects and background objects in the mask window are exactly equal, half and half. As mentioned above, the inter-class variance The maximum value is obtained, and in this case, the threshold calculated by Otsu is the most accurate, and no compensation value is required or only a very small compensation value is required to balance various errors. Therefore, in this case, the compensation value obtains the minimum value.

[0213] Based on the above analysis, the overall shape trend of the compensation value is a parabola with high ends and a low middle, opening upward. Based on the actual situation, the present invention only considers the situation where the full background is half of the background and the foreground is half of the background. This is a process in which the compensation value decreases. Therefore, for an underwater image, the minimum value of the compensation value δ is taken in the foreground window at the maximum central illumination, and the maximum value of the compensation value δ is taken in the background window at the edge. The value ranges of the coefficients a and c mentioned above are determined based on actual experiments. Generally speaking, the greater the turbidity, the smaller the value of a; the more obvious the light attenuation from the center to the edge of the image, the larger the value of c. The actual coefficient values ​​are taken according to this rule.

[0214] The adaptive classification segmentation of this implementation is summarized. The specific steps of the algorithm are as follows:

[0215] 1. First, convert the underwater image into a grayscale image and perform boundary expansion.

[0216] 2. Use the mask window to scan the entire image starting from the upper left corner of the original image.

[0217] 3. Analyze and process the grayscale values ​​of the pixels in the mask window. First, apply the EM algorithm to fit the grayscale histogram of the mask window. After fitting, determine the difference between the means of the two single Gaussian distributions in the mixed Gaussian model. If the difference is greater than the predetermined class spacing, the window is considered to be a foreground window, otherwise, the window is determined to be a background window.

[0218] 4. For the foreground window, the central pixel threshold = Otsu threshold - compensation value δ. The compensation value is calculated according to the specific underwater image conditions; for the background window, the central pixel threshold = the smaller mean of the two Gaussian distributions - 2.5 times the standard deviation of the distribution.

[0219] 5. Compare the grayscale value of the pixel in the center of the mask window with the corresponding threshold. If the pixel grayscale value is greater than the threshold, the corresponding pixel of the resulting binary image is set to 0, that is, black; otherwise, it is set to 255, that is, white.

[0220] 6. Determine whether the center pixel is the bottom left pixel. If so, end the process; otherwise, move right one pixel and repeat steps 3-5.

[0221] Figure 4 is a flowchart of image segmentation according to an embodiment of the present invention. The specific flowchart of the algorithm is as follows Figure 4As shown. The adaptive classification segmentation algorithm proposed in this embodiment starts from the disadvantages of global segmentation and focuses on the uneven illumination situation where the global segmentation algorithm fails. In view of different illumination conditions, a method of segmentation using the neighborhood grayscale information of the mask window is proposed. Although the global illumination distribution is uneven, the uneven illumination phenomenon is not serious when it is limited to a small neighborhood window, so it is effective to apply the classic global segmentation method in the neighborhood. In addition, since Otsu is only applicable to images with a double-peaked grayscale distribution histogram, this embodiment performs EM algorithm fitting on the grayscale distribution histogram before segmentation for the application condition of Otsu. Then, the difference between the means of the two single Gaussian distributions in the mixed Gaussian model is used as the inter-class distance to judge the nature of the mask window. Finally, since Otsu's threshold will be offset in the mask window with a large difference in the proportion of the two classes, this embodiment proposes a compensation value formula for the offset. This formula is derived based on experimental data and has strong robustness for different underwater illumination conditions and turbidity.

[0222] The adaptive classification and segmentation algorithm proposed in this embodiment can well separate underwater images according to different underwater lighting conditions and turbidity.

[0223] Figure 5 is a schematic diagram of the grid-segmentation result of the adaptive classification segmentation image according to an embodiment of the present invention, such as Figure 5 As shown, the left column (a)-(d) are the original grayscale images. It can be seen that the turbidity of the water is different. The water quality of (a) and (d) is good, and the images are clear. However, the water quality of (b) and (c) is poor, and the images are blurred. Correspondingly, the right side (e)-(h) is the result obtained by the adaptive classification segmentation algorithm. The results of (e) and (h) with better water quality are better, with less noise and clear foreground segmentation; while the water quality of (f) and (g) with poorer water quality has more noise. However, in general, the foreground objects are segmented more completely. Although (f) and (g) have noise, it does not affect subsequent processing. The coefficient pairs (a, c) of Figures (e)-(h) are (10,16), (8,7), (6,22), and (12,7) respectively. It can be seen that the more turbid the water, the smaller the δ compensation value. This is because the foreground and background are more difficult to separate when the water is turbid, so the difference between the proportion of the foreground and the proportion of the background is reduced, and the effect of the large-proportion background class on the threshold is weakened.

[0224] Figure 6 is a schematic diagram of an image segmentation device according to an embodiment of the present invention. Figure 6 As shown, according to another aspect of an embodiment of the present invention, an image segmentation device is provided, including: an acquisition module 60, a determination module 62, a segmentation module 64 and a traversal module 66. The device is described in detail below.

[0225] An acquisition module 60 is used to acquire a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of multiple areas of the target image, and the central pixel points of the multiple areas respectively correspond to the pixel points of the target image; a determination module 62 is connected to the above-mentioned acquisition module 60, and is used to determine the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; a segmentation module 64 is connected to the above-mentioned determination module 62, and is used to set the central pixel point to a first pixel value when the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold; when the pixel value of the central pixel point of the target area is less than the segmentation threshold, the central pixel point is set to a second pixel value, wherein the pixel difference between the first pixel value and the second pixel value is a preset threshold; a traversal module 56 is connected to the above-mentioned segmentation module 54, and is used to determine the first pixel value or the second pixel value corresponding to the multiple areas respectively by traversing the multiple areas, so as to generate a segmented image of the target image.

[0226] Through the above device, a pixel grayscale histogram of a target area of ​​a target image is obtained, wherein the target area is one of a plurality of areas of the target image, and the central pixels of the plurality of areas respectively correspond to the pixels of the target image; according to the pixel grayscale histogram of the target area, a corresponding segmentation threshold of the target area is determined; when the pixel value of the central pixel of the target area is greater than or equal to the segmentation threshold, the central pixel is set to a first pixel value; when the pixel value of the central pixel of the target area is less than the segmentation threshold, the central pixel is set to a second pixel value, wherein the pixel difference between the first pixel value and the second pixel value is a preset threshold; by traversing a plurality of areas, the first pixel value or the second pixel value respectively corresponding to the plurality of areas is determined, so as to generate a segmented image of the target image, the pixel grayscale histogram of the plurality of areas of the target image is obtained, the segmentation threshold of the target area is determined, and the target area is segmented, so as to achieve the purpose of segmenting the target image according to the target area corresponding to the pixel points, so that the segmentation of each pixel point depends on the pixel points around it, thereby achieving the technical effect of improving the accuracy of target image segmentation, and further solving the technical problem in the related art that the image segmentation accuracy is low due to the uneven illumination of the underwater segmented image and the block effect of the traditional segmentation method.

[0227] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-described image segmentation methods.

[0228] According to another aspect of an embodiment of the present invention, a computer storage medium is provided, the computer storage medium includes a stored program, wherein when the program is executed, the device where the computer storage medium is located is controlled to execute any one of the above-mentioned image segmentation methods.

[0229] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0230] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. Figure 7 As shown, an embodiment of the present invention provides an electronic device 10, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: obtaining a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of a plurality of areas of the target image, and the central pixels of the plurality of areas respectively correspond to the pixels of the target image; determining a corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; when the pixel value of the central pixel of the target area is greater than or equal to the segmentation threshold, setting the central pixel to a first pixel value; when the pixel value of the central pixel of the target area is less than the segmentation threshold, setting the central pixel to a second pixel value, and obtaining a compensated target area, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range; determining compensated areas corresponding to the plurality of areas respectively by traversing the plurality of areas, and combining all enhanced areas of the target image to generate a segmented image of the target image.

[0231] Optionally, obtaining a pixel grayscale histogram of a target area of ​​a target image includes: subtracting one from half of a preset size of a mask window to obtain an extended size for expanding the target image, wherein the mask window is square; expanding the target image according to the extended size to obtain an extended image after expansion, wherein the mask window is used to align pixel points of the target image through geometric centers and collect areas corresponding to the pixel points; traversing the extended image in a predetermined manner according to the mask window to obtain the multiple areas collected multiple times by the mask window, wherein the position of the mask window corresponds one-to-one to the areas; and counting the pixel values ​​of the pixel points of the target image contained in each of the areas to determine a pixel histogram of the area.

[0232] Optionally, determining the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area includes: determining the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value respectively; determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; when the target area is a foreground area, determining the first segmentation threshold corresponding to the foreground area by using the maximum inter-class difference algorithm; when the target area is a background area, determining the second segmentation threshold corresponding to the background area by using the first grayscale value and the second grayscale value and the standard deviation of the pixel grayscale histogram.

[0233] Optionally, before determining the first grayscale value and the second grayscale value with the highest frequency in the pixel grayscale histogram, it also includes: fitting a mixed Gaussian model to the pixel grayscale histogram of the target area to determine the initial value of the mean and the initial value of the covariance of the pixel grayscale histogram; selecting an initial value of the mixing coefficient; calculating the initial value of the likelihood function and the posterior probability based on the mean, the covariance and the initial value of the mixing coefficient; determining the updated mean, covariance and mixing coefficient based on the posterior probability as a weighting coefficient; determining the updated value of the likelihood function based on the updated mean, covariance and mixing coefficient; when the updated value of the likelihood function meets the convergence condition, updating the pixel grayscale histogram according to the corresponding mean and covariance; when the updated value of the likelihood function does not meet the convergence condition, continuing to execute using the mean, covariance and mixing coefficient corresponding to the updated value as the initial value.

[0234] Optionally, determining whether the target area is a background area or a foreground area based on the first grayscale value and the second grayscale value includes: subtracting the first grayscale value from the second grayscale value and taking the absolute value to obtain the inter-class distance; when the inter-class distance is greater than or equal to a first preset multiple of the sum of standard deviations, determining that the target area is a foreground area; when the inter-class distance is less than a first preset multiple of the sum of standard deviations, determining that the target area is a background area; wherein the sum of standard deviations is the sum of a first standard deviation corresponding to the first grayscale value and a second standard deviation corresponding to the second grayscale value.

[0235] Optionally, when the target area is a foreground area, after determining the first segmentation threshold corresponding to the foreground area by the maximum inter-class difference algorithm, it also includes: determining the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels and the average grayscale of the background pixels; exponentially taking the ratio of the inter-class variance to the first compensation coefficient as the base value of a natural constant, and multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of illumination attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area.

[0236] Optionally, when the area is a background area, determining the second segmentation threshold corresponding to the background area through the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram includes: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is used as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is used as the minimum value; obtaining the second segmentation threshold by subtracting the standard deviation of a second preset multiple from the minimum value, wherein the second preset multiple is determined according to a normal distribution; and using the second segmentation threshold as the segmentation threshold for the background area.

[0237] Optionally, after traversing the multiple regions to determine the compensated regions corresponding to the multiple regions respectively, and combining all enhanced regions of the target image to generate a segmented image of the target image, the method further includes: inputting the segmented image of the target image into a trained segmentation model, and the segmentation model outputs the segmented regions of the majority of the segmented images.

[0238] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0239] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of a plurality of areas of the target image, and the central pixels of the plurality of areas respectively correspond to the pixels of the target image; determining a corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area; setting the central pixel to a first pixel value when the pixel value of the central pixel of the target area is greater than or equal to the segmentation threshold; setting the central pixel to a second pixel value when the pixel value of the central pixel of the target area is less than the segmentation threshold, to obtain a compensated target area, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range; determining the compensated areas corresponding to the plurality of areas respectively by traversing the plurality of areas, and generating a segmented image of the target image by combining all enhanced areas of the target image.

[0240] Optionally, obtaining a pixel grayscale histogram of a target area of ​​a target image includes: subtracting one from half of a preset size of a mask window to obtain an extended size for expanding the target image, wherein the mask window is square; expanding the target image according to the extended size to obtain an extended image after expansion, wherein the mask window is used to align pixel points of the target image through geometric centers and collect areas corresponding to the pixel points; traversing the extended image in a predetermined manner according to the mask window to obtain the multiple areas collected multiple times by the mask window, wherein the position of the mask window corresponds one-to-one to the areas; and counting the pixel values ​​of the pixel points of the target image contained in each of the areas to determine a pixel histogram of the area.

[0241] Optionally, determining the corresponding segmentation threshold of the target area according to the pixel grayscale histogram of the target area includes: determining the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value respectively; determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; when the target area is a foreground area, determining the first segmentation threshold corresponding to the foreground area by using the maximum inter-class difference algorithm; when the target area is a background area, determining the second segmentation threshold corresponding to the background area by using the first grayscale value and the second grayscale value and the standard deviation of the pixel grayscale histogram.

[0242] Optionally, before determining the first grayscale value and the second grayscale value with the highest frequency in the pixel grayscale histogram, it also includes: fitting a mixed Gaussian model to the pixel grayscale histogram of the target area to determine the initial value of the mean and the initial value of the covariance of the pixel grayscale histogram; selecting an initial value of the mixing coefficient; calculating the initial value of the likelihood function and the posterior probability based on the mean, the covariance and the initial value of the mixing coefficient; determining the updated mean, covariance and mixing coefficient based on the posterior probability as a weighting coefficient; determining the updated value of the likelihood function based on the updated mean, covariance and mixing coefficient; when the updated value of the likelihood function meets the convergence condition, updating the pixel grayscale histogram according to the corresponding mean and covariance; when the updated value of the likelihood function does not meet the convergence condition, continuing to execute using the mean, covariance and mixing coefficient corresponding to the updated value as the initial value.

[0243] Optionally, determining whether the target area is a background area or a foreground area based on the first grayscale value and the second grayscale value includes: subtracting the first grayscale value from the second grayscale value and taking the absolute value to obtain the inter-class distance; when the inter-class distance is greater than or equal to a first preset multiple of the sum of standard deviations, determining that the target area is a foreground area; when the inter-class distance is less than a first preset multiple of the sum of standard deviations, determining that the target area is a background area; wherein the sum of standard deviations is the sum of a first standard deviation corresponding to the first grayscale value and a second standard deviation corresponding to the second grayscale value.

[0244] Optionally, when the target area is a foreground area, after determining the first segmentation threshold corresponding to the foreground area by the maximum inter-class difference algorithm, it also includes: determining the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels and the average grayscale of the background pixels; exponentially taking the ratio of the inter-class variance to the first compensation coefficient as the base value of a natural constant, and multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of illumination attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area.

[0245] Optionally, when the area is a background area, determining the second segmentation threshold corresponding to the background area through the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram includes: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is used as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is used as the minimum value; obtaining the second segmentation threshold by subtracting the standard deviation of a second preset multiple from the minimum value, wherein the second preset multiple is determined according to a normal distribution; and using the second segmentation threshold as the segmentation threshold for the background area.

[0246] Optionally, after traversing the multiple regions to determine the compensated regions corresponding to the multiple regions respectively, and combining all enhanced regions of the target image to generate a segmented image of the target image, the method further includes: inputting the segmented image of the target image into a trained segmentation model, and the segmentation model outputs the segmented regions of the majority of the segmented images.

[0247] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0248] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0249] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0250] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0251] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0252] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0253] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0254] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0255] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0256] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image segmentation method, characterized in that: include: Acquire a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of a plurality of areas of the target image, and central pixel points of the plurality of areas respectively correspond to pixel points of the target image; Determine a corresponding segmentation threshold of the target area according to a pixel grayscale histogram of the target area; When the pixel value of the central pixel point of the target area is greater than or equal to the segmentation threshold, the central pixel point is set to a first pixel value; when the pixel value of the central pixel point of the target area is less than the segmentation threshold, the central pixel point is set to a second pixel value, to obtain a compensated target area, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range; By traversing the multiple regions, determining the compensated regions respectively corresponding to the multiple regions, and combining all enhanced regions of the target image to generate a segmented image of the target image; Determining a corresponding segmentation threshold of the target area according to a pixel grayscale histogram of the target area includes: Determine the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value; Determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; In the case where the target area is a foreground area, a first segmentation threshold corresponding to the foreground area is determined by a maximum inter-class difference algorithm, including: calculating the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels, and the average grayscale of the background pixels; taking the ratio of the inter-class variance to the first compensation coefficient as an exponential, taking the value with a natural constant as the base, and multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of light attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area; In the case that the target area is a background area, the second segmentation threshold corresponding to the background area is determined by the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram, including: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is used as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is used as the minimum value; the second segmentation threshold is obtained by subtracting the standard deviation of a second preset multiple from the minimum value, wherein the second preset multiple is determined according to a normal distribution; and the second segmentation threshold is used as the segmentation threshold of the background area.

2. The method according to claim 1, characterized in that Obtaining the pixel grayscale histogram of the target area of ​​the target image includes: Subtracting one from half of the preset size of the mask window to obtain an extended size for expanding the target image, wherein the mask window is a square; The target image is expanded according to the expansion size to obtain an expanded image, wherein the mask window is used to align the pixel points of the target image through the geometric center and collect the area corresponding to the pixel points; Traversing the extended image in a predetermined manner according to the mask window, acquiring the multiple regions acquired by the mask window multiple times, wherein the positions of the mask window correspond one to one to the regions; Counting the pixel values ​​of the pixels of the target image contained in each of the regions to determine a pixel histogram of the region.

3. The method according to claim 1, characterized in that: Before determining the first grayscale value and the second grayscale value with the highest frequency in the pixel grayscale histogram, the method further includes: Fitting a mixed Gaussian model to a pixel grayscale histogram of the target area to determine an initial value of a mean and an initial value of a covariance of the pixel grayscale histogram; Select an initial value for the mixing coefficient; Calculating an initial value of a likelihood function and a posterior probability according to the mean, the covariance and the initial value of the mixing coefficient; Calculate the updated mean, covariance and mixing coefficient according to the posterior probability as a weighting coefficient; Calculate the updated value of the likelihood function based on the updated mean, covariance, and mixing coefficient; When the updated value of the likelihood function satisfies the convergence condition, updating the pixel grayscale histogram according to the corresponding mean and covariance; When the updated value of the likelihood function does not satisfy the convergence condition, the mean, covariance and mixing coefficient corresponding to the updated value are used as initial values ​​to continue the execution.

4. The method according to claim 1, characterized in that: Determining, according to the first grayscale value and the second grayscale value, whether the target area is a background area or a foreground area includes: Subtract the first grayscale value from the second grayscale value and take an absolute value to obtain an inter-class distance; When the inter-class distance is greater than or equal to a first preset multiple of the sum of standard deviations, determining the target area as a foreground area; When the inter-class distance is less than a first preset multiple of the sum of standard deviations, determining that the target area is a background area; The sum of the standard deviations is the sum of a first standard deviation corresponding to the first grayscale value and a second standard deviation corresponding to the second grayscale value.

5. The method according to any one of claims 1 to 4, characterized in that After traversing the multiple regions to determine compensated regions corresponding to the multiple regions respectively, and combining all enhanced regions of the target image to generate a segmented image of the target image, the method further includes: The segmented image of the target image is input into a trained segmentation model, and the segmentation model outputs the segmented regions of the majority of the segmented images.

6. An image segmentation device, characterized in that: include: An acquisition module, used for acquiring a pixel grayscale histogram of a target area of ​​a target image, wherein the target area is one of a plurality of areas of the target image, and central pixel points of the plurality of areas respectively correspond to pixel points of the target image; A determination module, used to determine a corresponding segmentation threshold of the target area according to a pixel grayscale histogram of the target area; A segmentation module, configured to, when a pixel value of a central pixel point of the target area is greater than or equal to the segmentation threshold, set the central pixel point to a first pixel value; when a pixel value of a central pixel point of the target area is less than the segmentation threshold, set the central pixel point to a second pixel value, to obtain a compensated target area, wherein a pixel difference value obtained by subtracting the first pixel value from the second pixel value is within a preset threshold range; A traversal module, configured to determine compensated regions respectively corresponding to the multiple regions by traversing the multiple regions, and generate a segmented image of the target image by combining all enhanced regions of the target image; The segmentation module is used for: Determine the first two grayscale values ​​with the highest frequency in the pixel grayscale histogram, which are the first grayscale value and the second grayscale value; Determining whether the target area is a background area or a foreground area according to the first grayscale value and the second grayscale value; In the case where the target area is a foreground area, a first segmentation threshold corresponding to the foreground area is determined by a maximum inter-class difference algorithm, including: calculating the inter-class variance according to the proportion of the number of foreground pixels in the foreground area, the proportion of the number of background pixels in the foreground area, the average grayscale of the foreground pixels, and the average grayscale of the background pixels; taking the ratio of the inter-class variance to the first compensation coefficient as an exponential, taking the value with a natural constant as the base, and multiplying it by the second compensation coefficient to obtain a compensation value, wherein the first compensation coefficient is the degree of light attenuation of the target image, and the second compensation coefficient is the cleanliness of the target image; adding the compensation value to the first segmentation threshold to determine the segmentation threshold corresponding to the foreground area; In the case that the target area is a background area, the second segmentation threshold corresponding to the background area is determined by the first grayscale value, the second grayscale value and the standard deviation of the pixel grayscale histogram, including: taking the minimum value of the first grayscale value and the second grayscale value, wherein if the first grayscale value is greater than the second grayscale value, the second grayscale value is used as the minimum value, and if the first grayscale value is less than the second grayscale value, the first grayscale value is used as the minimum value; the second segmentation threshold is obtained by subtracting the standard deviation of a second preset multiple from the minimum value, wherein the second preset multiple is determined according to a normal distribution; and the second segmentation threshold is used as the segmentation threshold of the background area.

7. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image segmentation method described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that The computer storage medium includes a stored program, wherein when the program is executed, the device where the computer storage medium is located is controlled to execute the image segmentation method according to any one of claims 1 to 5.