Image processing method and device, storage medium and electronic equipment
By adaptively adjusting the target threshold, the edge pixel points are determined according to the difference in image area, which solves the problem of poor image segmentation effect in the prior art, and achieves high-accuracy image segmentation under different conditions.
Patent Information
- Application Number
- CN202510422698.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-08-15
AI Technical Summary
The existing threshold method cannot adapt to different images to be processed in image segmentation, resulting in poor segmentation effect. Especially in the case of low-contrast images and poor lighting conditions, the segmentation between the foreground area and the background area is not obvious and difficult to accurately detect.
By adaptively adjusting the target threshold, the target threshold corresponding to the maximum difference is determined based on the difference degree of different regions in the image to be processed, which is used to determine whether the pixel point is an edge pixel point, thereby improving the accuracy of image segmentation.
It realizes that the edge pixel points can be accurately identified under different image conditions, improves the accuracy of image segmentation and the difference between the foreground area and the background area, and can effectively detect the foreground area under low contrast and poor lighting conditions.
Smart Images
Figure CN120495648A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, device, storage medium, and electronic device. Background Art
[0002] Currently, image segmentation technology exists, which is used to distinguish the foreground region of interest from the rest of the background in an image to be processed. For example, in a scenario where a learning machine is grading test papers, the machine needs to take a photo of the test paper to generate the image to be processed. It then detects and segments the edges of the test paper in the image to be processed, treating the test paper as the foreground region and the background region as the background region.
[0003] As one of the image segmentation technologies, the threshold method can segment the foreground area and the background area in the image to be processed. However, its segmentation effect is poor for different images to be processed. Summary of the Invention
[0004] The purpose of the present disclosure is to provide an image processing method, device, storage medium and electronic device to accurately identify edge pixels and improve the accuracy of image segmentation.
[0005] In order to achieve the above object, the present disclosure provides an image processing method, comprising:
[0006] For any specified threshold value among a plurality of specified threshold values, determining the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value, and the number of second pixels whose grayscale values are greater than the specified threshold value;
[0007] Obtaining a degree of difference corresponding to the specified threshold based on the first number of pixels and the second number of pixels; the degree of difference indicating a difference between a first region and a second region in the image to be processed;
[0008] A specified threshold corresponding to the maximum difference is selected from the differences corresponding to the multiple specified thresholds as a target threshold; the target threshold is used as a basis for determining whether a pixel point in the image to be processed is an edge pixel point, and the edge pixel point is used to segment the first area and the second area.
[0009] Optionally, obtaining the difference corresponding to the specified threshold according to the first number of pixels and the second number of pixels includes:
[0010] Determining a first pixel ratio of the first number of pixels to the total number of pixels in the image to be processed, and a second pixel ratio of the second number of pixels to the total number of pixels;
[0011] Determining a first grayscale value sum of pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold, and a second grayscale value sum of pixels whose grayscale values are greater than the specified threshold;
[0012] A degree of difference corresponding to the specified threshold is obtained according to the first pixel ratio, the second pixel ratio, the first grayscale value sum, and the second grayscale value sum.
[0013] Optionally, the method further includes:
[0014] Inputting the image to be processed into a first model;
[0015] For any pixel in the image to be processed, identifying, by the first model, a probability that the pixel is the edge pixel;
[0016] When the probability that the pixel point is the edge pixel point is greater than the target threshold, it is determined that the pixel point is the edge pixel point.
[0017] Optionally, the method further includes:
[0018] The second model is trained using multiple sets of training samples; each set of training samples includes a feature vector of a pixel sample extracted from an image sample by the first model and a classification result of the pixel sample; the classification result includes whether the pixel sample is an edge pixel or a non-edge pixel;
[0019] The feature vector of the pixel point extracted by the first model from the image to be processed is input into the trained second model to determine whether the pixel point is the edge pixel point or the non-edge pixel point.
[0020] Optionally, the method further includes:
[0021] Clustering the pixels in the image to be processed to obtain a plurality of third pixels and a plurality of fourth pixels; the plurality of third pixels are pixels in the first area, and the plurality of fourth pixels are pixels in the second area;
[0022] Determine a first average value of the grayscale values of the plurality of third pixels and a second average value of the grayscale values of the plurality of fourth pixels;
[0023] using a midpoint between the first average value and the second average value as an initial threshold value among the plurality of specified threshold values;
[0024] The initial threshold value and a threshold value whose difference from the initial threshold value is smaller than a preset value are used as the multiple designated threshold values.
[0025] Optionally, different images to be processed correspond to different target thresholds.
[0026] In order to achieve the above objectives, the present disclosure provides an image processing device, comprising:
[0027] a determination module configured to determine, for any specified threshold value among a plurality of specified threshold values, the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value, and the number of second pixels whose grayscale values are greater than the specified threshold value;
[0028] a difference module configured to obtain a difference corresponding to the specified threshold based on the first number of pixels and the second number of pixels, wherein the difference indicates a difference between a first region and a second region in the image to be processed;
[0029] The screening module is configured to screen out the specified threshold corresponding to the maximum difference from the differences corresponding to the multiple specified thresholds as the target threshold; the target threshold is used as a basis for determining whether a pixel point in the image to be processed is an edge pixel point, and the edge pixel point is used to segment the first area and the second area.
[0030] In order to achieve the above-mentioned objectives, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method proposed in the present disclosure.
[0031] In order to achieve the above objectives, the present disclosure provides an electronic device, comprising:
[0032] a memory having a computer program stored thereon;
[0033] A processor is used to execute the computer program in the memory to implement the steps of the image processing method proposed in the present disclosure.
[0034] In order to achieve the above-mentioned objective, the present disclosure provides a computer program product, including a computer program, which implements the steps of the image processing method proposed in the present disclosure when executed by a processor.
[0035] Through the above technical solution, for different images to be processed, it is possible to adaptively adjust the target threshold value that can make the difference between the first area and the second area of the image to be processed reach the maximum difference. Then, the accuracy of the edge pixel points obtained based on the target threshold value will be higher, and the difference between the first area and the second area divided by the dividing line composed of the edge pixel points will be greater, and the accuracy of the image segmentation will be better.
[0036] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:
[0038] Figure 1 It is a schematic diagram of dividing the test paper area and the non-test paper area according to an exemplary embodiment.
[0039] Figure 2 The figure is a schematic diagram of segmenting a portrait area and a non-portrait area according to an exemplary embodiment.
[0040] Figure 3 The figure is a flowchart of an image processing method according to an exemplary embodiment.
[0041] Figure 4 is a schematic diagram of a deep belief network model proposed according to an exemplary embodiment.
[0042] Figure 5 is a schematic diagram of a deep belief network model proposed according to an exemplary embodiment.
[0043] Figure 6 is a schematic diagram of a deep belief network model proposed according to an exemplary embodiment.
[0044] Figure 7 It is a schematic diagram of the linkage between a first model and a second model proposed according to an exemplary embodiment.
[0045] Figure 8 is a schematic diagram of clustering pixels into two categories according to an exemplary embodiment.
[0046] Figure 9 is a block diagram of an image processing apparatus according to an exemplary embodiment.
[0047] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment.
[0048] Figure 11 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0049] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.
[0050] Image segmentation is used to divide an image into a foreground area of current interest and a background area of current non-interest. After the foreground area is segmented, downstream processing such as target object recognition and target object classification can be performed on the foreground area.
[0051] Figure 1 It is a schematic diagram of image segmentation. Through image segmentation, Figure 1 The image to be processed is divided into the test paper area (foreground area) and the non-test paper area (background area). In this way, the answer area in the test paper area can be approved.
[0052] Figure 2 This is another schematic diagram of image segmentation. Through image segmentation, Figure 2 The image to be processed in the image processing apparatus is divided into a portrait area (foreground area) and a non-portrait area (background area). In this way, the face, limbs, etc. of each person in the portrait area can be recognized.
[0053] In image segmentation, a threshold method can be used. This method can identify edge pixels in the image to be processed. If the probability of a pixel being an edge pixel is greater than a threshold, the pixel is considered an edge pixel. Based on multiple edge pixels in the image to be processed, the dividing line between the foreground and background regions can be determined, and the image to be processed can be segmented into foreground and background regions using this dividing line. However, the threshold in the threshold method is a fixed threshold that cannot be adapted to different images to be processed. As a result, the foreground and background regions segmented based on the fixed threshold are incorrect, reducing the accuracy of image segmentation.
[0054] At least one embodiment of the present disclosure provides an image processing method, which finds a target threshold value that maximizes the difference between the first area and the second area of the image to be processed for different images to be processed. Then, for different images to be processed, the embodiment of the present disclosure adaptively adjusts the target threshold value of the image to be processed, so that the target threshold value can adapt to different images to be processed, thereby improving the accuracy of image segmentation.
[0055] Figure 3 It is a flowchart of the steps of an image processing method proposed by an exemplary embodiment of the present disclosure.
[0056] For example, the image processing method can be applied in a learning machine to segment the first area (such as the test paper area) and the second area (such as the non-test paper area) in the image to be processed scanned by the learning machine; it can also be applied in mobile phones, computers, notebooks and other terminals to segment the first area (such as the person area) and the second area (such as the non-person area) in the acquired image to be processed, etc. Of course, it can also be applied to other electronic devices that require image segmentation, and the present disclosure does not limit this.
[0057] like Figure 3 As shown, in at least one embodiment, the image processing method includes the following steps.
[0058] In step S10 , for any specified threshold value among the multiple specified threshold values, the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value and the number of second pixels whose grayscale values are greater than the specified threshold value are determined.
[0059] The specified threshold is a preset threshold used to determine whether a pixel in the image being processed is an edge pixel. If the probability that a pixel in the image being processed is an edge pixel is greater than the specified threshold, the pixel is considered an edge pixel. If the probability that a pixel in the image being processed is an edge pixel is less than the specified threshold, the pixel is considered a non-edge pixel.
[0060] If the pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold are located in the first area, then the first number of pixels is the total number of pixels in the first area, and is also the total number of pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold; if the pixels in the image to be processed whose grayscale values are greater than the specified threshold are located in the second area, then the second number of pixels is the total number of pixels in the second area, and is also the total number of pixels in the image to be processed whose grayscale values are greater than the specified threshold.
[0061] The first region and the second region constitute the image to be processed. If the first region is the foreground region of the image to be processed, the second region is the background region of the image to be processed; conversely, if the first region is the background region of the image to be processed, the second region is the foreground region of the image to be processed.
[0062] The specified threshold is any grayscale value among multiple grayscale values of the pixel point. For example, if the grayscale value of the pixel point is within the range of [0, 255], the corresponding specified threshold will also be screened within the range of [0, 255] and is any grayscale value within the grayscale range of [0, 255].
[0063] In step S20, a difference degree corresponding to a specified threshold is obtained based on the first number of pixels and the second number of pixels. The difference degree indicates the difference between the first region and the second region in the image to be processed.
[0064] The difference between the first area and the second area under the specified threshold can be obtained based on the number of first pixels and the number of second pixels. The larger the difference, the more obvious the difference between the first area and the second area after segmentation by the specified threshold, and the more accurate the specified threshold is.
[0065] The difference between the first region and the second region can be reflected by the inter-class variance. The larger the inter-class variance is, the greater the difference between the first region and the second region is.
[0066] In step S30 , a designated threshold corresponding to the maximum difference is selected from the differences corresponding to the multiple designated thresholds as a target threshold.
[0067] The target threshold is used to determine whether a pixel in the image being processed is an edge pixel. If the probability that a pixel in the image being processed is an edge pixel is greater than the target threshold, the pixel is considered an edge pixel. If the probability that a pixel in the image being processed is an edge pixel is less than the target threshold, the pixel is considered a non-edge pixel.
[0068] Optionally, iterative calculation can be performed. When the difference corresponding to the previous specified threshold does not meet the convergence condition, the difference corresponding to the next specified threshold is calculated until the obtained difference meets the convergence condition. Only then will the specified threshold corresponding to the difference when the convergence condition is met be used as the target threshold.
[0069] The convergence conditions include: the difference no longer continues to increase or the difference is greater than or equal to a preset value.
[0070] It can be understood that in image segmentation technology, the difference between each pixel in the foreground area is small, and the difference between each pixel in the background area is also small, while the difference between the foreground area and the background area is usually large. The present disclosure uses this feature to determine a target threshold that can make the difference between the foreground area and the background area of the image to be processed reach the maximum difference. The segmentation line composed of edge pixels screened by the target threshold can make the difference between the foreground area and the background area reach the maximum difference. Then the foreground area and background area divided based on the target threshold will be more accurate.
[0071] Through the above technical solution, a target threshold value that can make the difference between the first area and the second area of the image to be processed reach the maximum difference can be determined for different images to be processed. Then, the difference between the first area and the second area divided based on the target threshold value will be greater, and the accuracy of the image segmentation will be better.
[0072] In the related technology, the processing capability of low-contrast images is weak. Under poor lighting conditions, the contrast of the image to be processed will be low, and the dividing line between the foreground area and the background area in the image to be processed will not be obvious, making it difficult to detect the foreground area.
[0073] The disclosed embodiment obtains the target threshold corresponding to when the difference between the foreground area and the background area reaches the maximum difference. After the edge pixel points are identified by the target threshold, the difference between the foreground area and the background area on both sides of the dividing line composed of the edge pixel points is large, and the foreground area can be detected even if the contrast of the image to be processed is not high.
[0074] According to an exemplary embodiment, an exemplary solution for obtaining the difference corresponding to the specified threshold in step S20 is shown. The exemplary solution includes the following steps:
[0075] (1) Determine a first pixel ratio of the first number of pixels to the total number of pixels in the image to be processed, and a second pixel ratio of the second number of pixels to the total number of pixels.
[0076] The cumulative distribution function (CDF) can be used to count the cumulative number of pixels at each grayscale value in multiple grayscale values, and then the grayscale value with the same value as the specified threshold is screened out from the multiple grayscale values, and the cumulative number of pixels at the grayscale value with the same value is used as the first number of pixels in the image to be processed whose grayscale value is less than or equal to the specified threshold.
[0077] For example, if the grayscale value is in the range of [0, 255], if the calculated cumulative number of pixels under the grayscale value of 250 is 1000, and the specified threshold is also 250, then the number of the first pixels in the image to be processed whose grayscale value is less than or equal to the specified threshold 250 is also 1000.
[0078] The grayscale values can be arranged in ascending order, from 0 to 255, and the cumulative number of pixels under a grayscale value is the sum of the number of pixels under the grayscale value and the number of pixels under the remaining grayscale values below the grayscale value.
[0079] For example, arrange the grayscale values in ascending order, from 0 to 255. If the number of pixels under the grayscale value of 0 is 10, and the number of pixels under the grayscale value of 1 is 20, then the cumulative number of pixels under the grayscale value of 1 is 30; similarly, if the number of pixels under the grayscale value of 2 is 50, then the cumulative number of pixels under the grayscale value of 2 is 80. By accumulating in this way, the cumulative number of pixels under different grayscale values can be obtained.
[0080] By using the cumulative distribution function to calculate the cumulative number of pixels at each grayscale value among multiple grayscale values, when determining the first number of pixels in the image to be processed whose grayscale value is less than or equal to the specified threshold, the grayscale value with the same specified threshold can be directly screened out from the multiple grayscale values, and the cumulative number of pixels of the grayscale value can be used as the first number of pixels, without the need to temporarily calculate the first number of pixels, thereby improving the speed of obtaining the first number of pixels.
[0081] The first area and the second area constitute the area to be processed, so the sum of the number of first pixels in the first area and the number of second pixels in the second area is equal to the total number of pixels in the image to be processed. After obtaining the number of first pixels, the number of first pixels can be subtracted from the total number to obtain the number of second pixels with a grayscale value greater than the specified threshold.
[0082] After obtaining the first number of pixels, the first number of pixels may be divided by the total number of pixels to obtain a first pixel ratio; after obtaining the second number of pixels, the second number of pixels may be divided by the total number of pixels to obtain a second pixel ratio.
[0083] (2) Determine the sum of the first grayscale values of the pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold, and the sum of the second grayscale values of the pixels whose grayscale values are greater than the specified threshold.
[0084] The grayscale values of pixels in the image to be processed whose grayscale values are less than or equal to a specified threshold can be added to obtain a first grayscale value sum; the grayscale values of pixels whose grayscale values are greater than the specified threshold can be added to obtain a second grayscale value sum.
[0085] (3) Obtaining a degree of difference corresponding to the specified threshold value according to the first pixel ratio, the second pixel ratio, the first grayscale value sum, and the second grayscale value sum.
[0086] The first average grayscale value can be obtained based on the sum of the first grayscale values and the first number of pixels; the second average grayscale value can be obtained based on the sum of the second grayscale values and the second number of pixels; and then the difference corresponding to the specified threshold is obtained based on the first pixel ratio, the second pixel ratio, the first average grayscale value, the second average grayscale value and the average grayscale value of all pixels in the image to be processed.
[0087] The first average grayscale value may be a ratio of the sum of the first grayscale values of pixels in the image to be processed whose grayscale values are less than or equal to a specified threshold to the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold.
[0088] The second average grayscale value may be obtained by taking the ratio of the sum of the second grayscale values of the pixels in the image to be processed whose grayscale values are greater than the specified threshold to the second number of pixels in the image to be processed whose grayscale values are greater than the specified threshold.
[0089] According to the first pixel ratio, the second pixel ratio, the first average grayscale value, the second average grayscale value, and the average grayscale value of all pixels in the image to be processed, the difference corresponding to the specified threshold can be obtained by the following formula:
[0090]
[0091] In formula (1), is the difference corresponding to the specified threshold; w0 is the first pixel ratio; w1 is the second pixel ratio; μ0 is the first average grayscale value; μ1 is the second average grayscale value; μ is the average grayscale value of all pixels in the image to be processed.
[0092] It can be seen from the above formula (1) that the first pixel ratio, the second pixel ratio, the first average grayscale value and the second average grayscale value are all in direct proportion to the difference between the specified threshold value.
[0093] Through the above technical solution, the difference corresponding to the specified threshold can be calculated through the first number of pixels, the second number of pixels, the first average grayscale value and the second average grayscale value. For example, the first number of pixels and the second number of pixels are used to obtain the first pixel ratio and the second pixel ratio respectively, and then the first pixel ratio, the second pixel ratio, the first average grayscale value and the second average grayscale value are used to obtain the difference corresponding to the specified threshold.
[0094] In this process, after obtaining each specified threshold, the first number of pixels and the second number of pixels can be screened out by the specified threshold, and then the first pixel ratio and the second pixel ratio are calculated based on the first number of pixels and the second number of pixels. The first pixel ratio and the second pixel ratio are then input into the above formula (1) to obtain the difference corresponding to the specified threshold, that is, the inter-class variance. The specified threshold that can make the inter-class variance reach the maximum value is then reversely deduced through the inter-class variance, and the specified threshold is used as the target threshold.
[0095] Figures 3 to 6An exemplary solution involved in the present disclosure is shown, which is used to explain the steps of identifying edge pixels based on a target threshold, and includes the following steps.
[0096] (1) Input the image to be processed into the first model.
[0097] The first model is used to identify the probability that a pixel point in the input image to be processed is an edge pixel point based on the input image to be processed.
[0098] The first model can be a Deep Belief Network (DBN), which is composed of a stack of multiple layers of Restricted Boltzmann Machines (RBMs). The Boltzmann Machine model is composed of multiple layers of networks, each of which is divided into explicit layers and hidden layers. The explicit layers receive the input image to be processed and transmit it to the hidden layers, which extract feature vectors of pixels in the image.
[0099] The deep belief model includes at least one Boltzmann machine model, and the included Boltzmann machine model includes at least one explicit layer and at least one hidden layer.
[0100] For example, see Figure 4 As shown in the figure, the deep belief model includes a Boltzmann machine model. Boltzmann machine models all consist of one explicit layer and three hidden layers. The connection between the top third hidden layer and the second hidden layer is bidirectional, forming an associative memory. Associative memory refers to associating the second hidden layer with the third hidden layer. After obtaining the feature vector extracted by the second hidden layer, the feature vector extracted by the third hidden layer can be associated with it, and vice versa. The connections between the second hidden layer and the first hidden layer, and between the first hidden layer and the underlying explicit layer, are unidirectional.
[0101] For example, see Figure 5 As shown in the figure, the deep belief model consists of two Boltzmann machine models, each of which consists of an explicit layer and a hidden layer. After receiving the image to be processed, the explicit layer at the bottom transmits it to the hidden layer, which extracts the feature vector of the pixels in the image. The hidden layer then transmits this feature vector to the top explicit layer, which transmits it to the top hidden layer. The top hidden layer then performs a second feature extraction on the feature vector.
[0102] The steps of training the Boltzmann machine model in the first model include:
[0103] In step A1, each network layer in the Boltzmann machine model is trained separately and unsupervised.
[0104] For example, the network parameters (such as weights and biases) of each network layer in the Boltzmann machine model can be initialized first. The weights can be randomly initialized, and the biases can be set to zero or a small random value. Then, the image samples are input into the input layer, and the input layer passes the image samples to at least one hidden layer. The at least one hidden layer converts the image samples into feature vectors and inputs them into the input layer of the next Boltzmann machine model.
[0105] During this process, the calculation of each network layer in the Boltzmann machine model includes the following sub-steps:
[0106] Step A11: input the image sample into the input layer.
[0107] Step A12: Calculate the activation probability of the hidden layer, and determine the activation state of the hidden layer according to the calculated activation probability, where the activation state is 0 or 1.
[0108] Step A13, reconstructing the state of the input layer based on the activation state of the hidden layer and the transpose of the weight matrix.
[0109] Step A14: Obtain a reconstruction error based on the state of the input layer after reconstruction and the state of the input layer before reconstruction, and update the gradient of the weight and the gradient of the bias based on the reconstruction error.
[0110] Repeat the above steps until the updated weights and biases change little or the number of iterations is reached, at which point the target weights and biases are obtained.
[0111] Step A2: A back propagation network (BP) is set in the last hidden layer of the deep belief network model, and the BP network is used as a classifier to obtain the classification results of the pixels in the image to be processed. The BP network receives the feature vector output by the last network layer of the deep belief network model, and uses the feature vector as the training parameter of the BP network, so that after the BP network is trained, it can output the classification results of the pixels.
[0112] For example, the back propagation network can predict the predicted probability that a pixel in the image to be processed is an edge pixel. When training the back propagation network, the loss value between the predicted probability that the pixel output by the back propagation network is an edge pixel and the actual probability that the pixel is an edge pixel can be used to fine-tune the target weight and target bias obtained in step A14, so that the predicted probability predicted by the back propagation network can gradually approach the actual probability.
[0113] In the above steps, step A1 is the pre-training process of the first model, which uses the state before and after the reconstruction of the input layer to update the weights and biases; step A2 is the fine-tuning process of the first model, which uses the loss value between the predicted probability and the true probability output by the back-propagation network to fine-tune the weights and biases. In this way, the first model finally obtained can predict the accurate probability that the pixel is an edge pixel.
[0114] The following formula will illustrate the training optimization process of the first model:
[0115] like Figure 6 As shown in Figure a, suppose there is a Boltzmann machine model, which includes a dominant layer and a hidden layer, such as Figure 6 As shown in Figure b, based on this Boltzmann machine model, a deep belief network model with a dominant layer and two hidden layers can be constructed. Figure 6 In Figure b, the weight W of the second hidden layer (2) is the weight W of the explicit layer (1) The transpose of The number of units in the explicit layer and the second hidden layer is the same, both are 5. The joint probability distribution between the explicit layer and the first hidden layer in the deep belief network model is: P(v, h (1) , h (2) ;θ)=P(v|h (1) ;W (1) )P(h (1) , h (2) ;W (2) , the joint probability is divided into two parts, one part is the directed belief network θ=W (1) , W (2) , P(v|h (1) ;W (1) ), the other part is the joint distribution P(h (1) ,h (2) ;W (2) ).
[0116] The calculation formula for the conditional probability distribution in the joint probability distribution between the explicit layer and the first hidden layer is as follows:
[0117]
[0118] In the above formula (2), h (1) is the variable input to the first hidden layer, which may contain multiple pixel points y i ;W (1) are the weights of the first hidden layer.
[0119] Among them, the joint distribution P(h(1) ,h (2) ;W (2) ) is calculated as follows:
[0120]
[0121] In the above formula (3), h (1) is the variable input to the first hidden layer; h (2) is the variable input to the second hidden layer; W (2) are the weights of the second hidden layer.
[0122] For each network layer in the deep belief network model, a layer-by-layer greedy strategy is adopted, which requires the following prerequisites:
[0123] First premise, the joint probability distribution of the explicit layer and the first hidden layer in the deep belief network model The joint probability distribution P(v, h) between the explicit layer and the first hidden layer in the Boltzmann machine model (1) ;W (1) )same.
[0124] The second premise is that without considering the explicit layer, the marginal distribution P(v, h) of the first hidden layer of the deep belief network model and the Boltzmann machine model is (1) |W (1) ) is the same.
[0125] On the basis of guaranteeing the above two premises, the weight of the second hidden layer can be improved W (2) That is, on the basis of ensuring that the performance of the deep belief network model is the same as or even better than that of the Boltzmann machine model, the weight W of the second hidden layer in the deep belief network model can be adjusted. (2) Make improvements.
[0126] The estimated inference value Q(h (1) |v), and use the simulated expectation maximization (EM) algorithm to find the log-likelihood function log P(v;θ) of the deep belief network model. The calculation formula is as follows:
[0127]
[0128] In the above formula (4), the first two formulas show the introduction of the estimated inference value Q(h (1) |v) to get the minimum value of the log-likelihood function log P(v;θ), and the greedy strategy is to fix the weight W of the first hidden layer. (1) , and maximize the minimum value of the log-likelihood function log P(v;θ), that is, maximize ∑Q(h(1) |v)log P(h (1) |W (2) ).
[0129] In maximizing ∑Q(h (1) |v)logP(h (1) |W (2) ) After that, we can find the weight W of the second hidden layer that maximizes the log-likelihood function (2) .
[0130] (2) For any pixel point in the image to be processed, the probability that the pixel point is the edge pixel point is identified by the first model.
[0131] After the first model is trained, the image to be processed can be input into the first model, and the probability that a pixel in the image to be processed is an edge pixel can be identified by the classifier in the first model.
[0132] (3) When the probability that the pixel point is the edge pixel point is greater than the target threshold, determine that the pixel point is the edge pixel point.
[0133] When the probability that a pixel point is an edge pixel point is greater than the target threshold, it indicates that the probability that the pixel point is an edge point is relatively high, and therefore it can be determined that the pixel point is an edge pixel point.
[0134] When the probability that a pixel is an edge pixel is less than or equal to the target threshold, it indicates that the pixel is a non-edge pixel, for example, a pixel located in the first area or in the second area.
[0135] Through the above technical solution, by training the first model, after the first model is trained, the pixel points of the image to be processed can be input into the first model to determine the probability that the pixel point is an edge pixel point.
[0136] Figure 7 An exemplary embodiment of the present disclosure is shown, which is used to explain that when the classification result obtained is not effective, the classification result can be obtained again by using a second model, and the exemplary embodiment includes the following steps:
[0137] (1) Use multiple sets of training samples to train the second model.
[0138] Each group of training samples includes feature vectors of pixel samples extracted from image samples by the first model and classification results of the pixel samples.
[0139] The first model is the aforementioned deep belief model. The deep belief network model can extract features from image samples to obtain feature vectors of pixel samples in the image samples. The feature vectors include the grayscale value, texture information, position information, shape information, and the similarity between adjacent pixel samples. The deep learning model can also identify pixel samples in the image samples to obtain classification results for the pixel samples. The classification results include whether the pixel samples are edge pixels or non-edge pixels. For example, the probability of identifying a pixel sample as an edge pixel is greater than a target threshold, and the pixel sample is considered to be an edge pixel, thereby obtaining a classification result for the pixel sample.
[0140] Therefore, when training the second model, the feature vector of the pixel sample extracted by the first model can be used as the input parameter of the second model, and the classification result of the pixel sample output by the first model, for example, whether the pixel sample is an edge pixel or a non-edge pixel, can be used as the label of the second model to train the second model, so that the second model can obtain the classification result of the pixel based on the feature vector of the pixel input subsequently.
[0141] The second model can be a conditional random field (CRF) model, which can predict whether the pixel is an edge pixel or a non-edge pixel based on the grayscale value, texture information, position information, shape information, similarity between adjacent pixel samples, and other information of the pixel sample.
[0142] (2) The feature vector of the pixel point extracted from the image to be processed by the first model is input into the trained second model to determine whether the pixel point is an edge pixel point or a non-edge pixel point.
[0143] When applying the second model, the feature vectors of the pixels extracted by the first model from the image to be processed, such as grayscale value, texture information, position information, shape information, similarity between adjacent pixel samples, etc., can be input into the second model to accurately identify whether the pixel is an edge pixel or a non-edge pixel.
[0144] The first model determines that a pixel is an edge pixel only when the probability that the pixel is an edge pixel is greater than the target threshold. However, this method of using the target threshold to distinguish whether a pixel is an edge pixel is uncertain. For images to be processed with lighting changes, noise or poor image quality, pixels in areas with obvious lighting changes and obvious noise will be mistakenly judged as edge pixels, resulting in inaccurate image segmentation. In addition, the method of using the target threshold to distinguish whether a pixel is an edge pixel will also cause the dividing line between the identified first area and the second area to be discontinuous or broken.
[0145] Similarly, the Sobel operator is based on the first-order derivative of an image. The first-order derivative is the rate of change of brightness or color of the image being processed in the horizontal or vertical direction. The Sobel operator first selects a window and calculates the horizontal or vertical rate of change of color or brightness for each pixel within the window. When the color or brightness change rate is large, it indicates that these pixels are edge pixels of the image being processed. However, if the image being processed has brightness changes caused by shooting lighting problems, the color or brightness change rate of these pixels will be large, and these pixels will be incorrectly identified as edge pixels.
[0146] By learning the feature vectors such as the grayscale value, texture information, position information, shape information, and similarity between adjacent pixel samples extracted by the first model through the second model, it can cope with such suddenly changing scenes. For example, for pixels with obvious lighting changes, obvious noise, complex textures, and complex backgrounds, although the probability that the pixels are edge pixels is greater than the target threshold, the similarity between these pixels and adjacent pixels is large. Under normal circumstances, the similarity between edge pixels and adjacent pixels is small, indicating that these pixels are not edge pixels, but pixel changes caused by reasons such as lighting changes, noise, complex textures, and complex backgrounds. In this case, these pixels will not be misjudged as edge pixels. In addition, the second model encourages the smoothness of the label field through regularization terms, which helps to generate more coherent segmentation lines.
[0147] In the related art, the detection effect on curved or broken edges is also poor. Some algorithms have good detection effect on straight edges, but poor detection effect on curved or broken edges.
[0148] By learning the feature vectors of each pixel, such as the grayscale value, texture information, position information, shape information, and similarity between adjacent pixel samples, the second model can fully learn the feature vectors of each pixel at multiple scales, thereby more accurately judging these curved or broken edge pixels.
[0149] Figure 8An exemplary solution involved in the present disclosure is shown, which is used to explain an exemplary solution for obtaining various specified thresholds, and includes the following steps:
[0150] (1) Clustering the pixels in the image to be processed to obtain a plurality of third pixels and a plurality of fourth pixels.
[0151] The plurality of third pixel points are pixel points in the first area, and the plurality of fourth pixel points are pixel points in the second area.
[0152] For example, taking the example that the first area is the foreground area in the image to be processed and the second area is the background area in the image to be processed, multiple pixel points in the image to be processed can be clustered, and the pixel points in the image to be processed can be divided into a third pixel point located in the foreground area and a fourth pixel point located in the background area.
[0153] The clustering method includes the following steps:
[0154] Step B1, setting the initial value of clustering.
[0155] You can set the desired number of categories, see for example Figure 8 As shown, the number of categories can be set to 2, and the expected number of categories after clustering is 2; the number of cluster centers can also be set; the minimum number of pixels in each category; the maximum number of pixels in each category; the maximum value of the standard deviation of the distance of each component distribution in each column; the minimum value of the distance between the cluster centers of two categories; the clustering stopping condition; the allowed clustering time, etc.
[0156] Step B2: Initialize the cluster centers, calculate the fuzzy membership of each pixel and the center vector of each cluster center.
[0157] Fuzzy membership is used to indicate the degree to which a pixel belongs to a certain category. A pixel can belong to multiple categories at the same time, and the fuzzy membership of a pixel in different categories is different. For example, a pixel belongs to category 1 with a fuzzy membership of 0.7 and to category 2 with a fuzzy membership of 0.3. The closer the distance between the pixel and the cluster center, the higher the fuzzy membership of the pixel to the category corresponding to the cluster center.
[0158] The central vector of the cluster center is the mathematical representation of the cluster center, and the central vector is the average value of all pixels belonging to the category where the cluster center is located.
[0159] The Fuzzy C-Means Clustering (FCM) algorithm can be used to continuously adjust the fuzzy membership and the center vector of the cluster center until a stopping condition is met. For example, a center vector can be randomly selected. For each pixel, the distance from the pixel to the cluster center is calculated, and the fuzzy membership of the pixel is calculated based on the distance. The pixel membership is then used to update the center vector of each cluster center, and the above steps are repeated until the stopping condition is met.
[0160] Step B3, calculate the number of pixels in each category. If the number of pixels in the category is less than the first preset value, delete the cluster center of the category and return to the above step B2. If the number of pixels in the category is greater than or equal to the first preset value, determine the number of cluster centers. If the number of cluster centers is less than half of the expected number of categories, perform a classification operation on the cluster centers and split the cluster centers into two cluster centers. If the number of cluster centers is greater than twice the expected number of categories, perform a merge operation on the cluster centers and merge the two cluster centers into one cluster center. If the number of cluster centers is greater than or equal to half of the expected number of categories and less than or equal to twice the expected number of categories, determine whether the number of cluster centers is odd or even. If it is an odd number, perform the above-mentioned splitting operation; if it is an even number, perform the above-mentioned merging operation.
[0161] It can be understood that the above stopping condition is that the number of cluster centers no longer changes and no splitting or merging operations are performed on the cluster centers.
[0162] (2) Determine a first average value of the grayscale values of the plurality of third pixel points and a second average value of the grayscale values of the plurality of fourth pixel points.
[0163] A first average value of the grayscale values of the plurality of third pixels in the first area and a second average value of the grayscale values of the plurality of pixels in the second area may be determined.
[0164] For example, taking the first area as the foreground area in the image to be processed and the second area as the background area in the image to be processed, the first average grayscale value of multiple third pixel points in the foreground area and the second average grayscale value of multiple pixel points in the background area can be calculated.
[0165] (3) The midpoint between the first average value and the second average value is used as the initial threshold value among the multiple specified threshold values.
[0166] For example, if the first average value is 24 and the second average value is 200, 112 may be used as the initial threshold value.
[0167] (4) The initial threshold and the threshold whose difference from the initial threshold is smaller than a preset value are used as multiple designated thresholds.
[0168] The multiple designated thresholds may include an initial threshold, a threshold that is above the initial threshold and whose difference from the initial threshold is smaller than a preset value, and a threshold that is below the initial threshold and whose difference from the initial threshold is smaller than a preset value.
[0169] For example, if the initial threshold is 112 and the preset value is 100, then the grayscale values within the range of 12 to 212 can be used as the designated threshold.
[0170] Through the above technical solution, the third pixel point in the first area and the fourth pixel point in the second area can be distinguished by clustering, and the initial threshold can be obtained based on the third pixel point and the fourth pixel point as a reference for threshold adjustment, rather than adjusting the randomly selected threshold, so that the threshold adjustment is more directional.
[0171] Figure 9 is a block diagram of an image processing apparatus 900 according to an exemplary embodiment. The image processing apparatus 900 includes: a judgment module 910 , a difference module 920 , and a screening module 930 .
[0172] The determination module 910 is configured to determine, for any specified threshold value among a plurality of specified threshold values, the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value, and the number of second pixels whose grayscale values are greater than the specified threshold value;
[0173] a difference module 920 configured to obtain a difference corresponding to the specified threshold based on the first number of pixels and the second number of pixels; the difference indicating a difference between the first region and the second region in the image to be processed;
[0174] The screening module 930 is configured to screen out the specified threshold corresponding to the maximum difference from the differences corresponding to the multiple specified thresholds as the target threshold; the target threshold is used as a basis for determining whether the pixel point in the image to be processed is an edge pixel point, and the edge pixel point is used to segment the first area and the second area.
[0175] Optionally, the difference module 920 includes:
[0176] a pixel ratio submodule, configured to determine a first pixel ratio of the first number of pixels to the total number of pixels in the image to be processed, and a second pixel ratio of the second number of pixels to the total number of pixels;
[0177] a grayscale value sum submodule, configured to determine a first grayscale value sum of pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold, and a second grayscale value sum of pixels whose grayscale values are greater than the specified threshold;
[0178] The difference submodule is configured to obtain a difference corresponding to the specified threshold according to the first pixel ratio, the second pixel ratio, the first grayscale value sum, and the second grayscale value sum.
[0179] Optionally, the image processing apparatus 900 further includes:
[0180] An input module, configured to input the image to be processed into the first model;
[0181] an identification module configured to identify, for any pixel in the image to be processed, a probability that the pixel is the edge pixel using the first model;
[0182] The first prediction module is configured to determine that the pixel point is the edge pixel point when the probability that the pixel point is the edge pixel point is greater than the target threshold.
[0183] Optionally, the image processing apparatus 900 further includes:
[0184] a training module configured to train the second model using multiple sets of training samples; each set of training samples includes a feature vector of a pixel sample extracted from an image sample by the first model and a classification result of the pixel sample; the classification result includes whether the pixel sample is an edge pixel or a non-edge pixel;
[0185] The second prediction module is configured to input the feature vector of the pixel point extracted by the first model from the image to be processed into the trained second model to determine whether the pixel point is the edge pixel point or the non-edge pixel point.
[0186] Optionally, the image processing apparatus 900 further includes:
[0187] a clustering module configured to cluster the pixels in the image to be processed to obtain a plurality of third pixels and a plurality of fourth pixels; the plurality of third pixels are pixels in the first area, and the plurality of fourth pixels are pixels in the second area;
[0188] an average value module, configured to determine a first average value of the grayscale values of the plurality of third pixels and a second average value of the grayscale values of the plurality of fourth pixels;
[0189] an initial threshold module, configured to use a midpoint between the first average value and the second average value as an initial threshold value among the plurality of specified threshold values;
[0190] The designated threshold module is configured to use the initial threshold and a threshold whose difference with the initial threshold is smaller than a preset value as the multiple designated thresholds.
[0191] Optionally, different images to be processed correspond to different target thresholds.
[0192] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0193] Figure 10 FIG. 1 is a block diagram of an electronic device 1000 according to an exemplary embodiment. Figure 10 As shown, the electronic device 1000 may include: a processor 1001 , a memory 1002 , and may further include one or more of a multimedia component 1003 , an input / output (I / O) interface 1004 , and a communication component 1005 .
[0194] The processor 1001 is used to control the overall operation of the electronic device 1000 to complete all or part of the steps in the above-mentioned image processing method. The memory 1002 is used to store various types of data to support the operation of the electronic device 1000. For example, this data may include instructions for any application or method operating on the electronic device 1000, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 1002 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 1003 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 1002 or sent via the communication component 1005. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 1004 provides an interface between the processor 1001 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 1005 is used for wired or wireless communication between the electronic device 1000 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 1005 may include: a Wi-Fi module, a Bluetooth module, an NFC module.
[0195] In an exemplary embodiment, the electronic device 1000 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned image processing method.
[0196] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-described image processing method. For example, the computer-readable storage medium may be the aforementioned memory 1002 including the program instructions. The program instructions may be executed by the processor 1001 of the electronic device 1000 to perform the above-described image processing method.
[0197] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a processor, and the computer program implements the steps of the above-mentioned image processing method when executed by the processor.
[0198] Figure 11 1 is a block diagram of an electronic device 1100 according to an exemplary embodiment. For example, the electronic device 1100 may be provided as a server. Figure 11 The electronic device 1100 includes a processor 1122, which may be one or more, and a memory 1132 for storing a computer program executable by the processor 1122. The computer program stored in the memory 1132 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 1122 may be configured to execute the computer program to perform the above-mentioned image processing method.
[0199] In addition, the electronic device 1100 may further include a power supply component 1126 and a communication component 1150. The power supply component 1126 may be configured to perform power management of the electronic device 1100, and the communication component 1150 may be configured to implement communication, such as wired or wireless communication, of the electronic device 1100. In addition, the electronic device 1100 may further include an input / output (I / O) interface 1158. The electronic device 1100 may operate based on an operating system stored in the memory 1132.
[0200] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-described image processing method. For example, the computer-readable storage medium may be the aforementioned memory 1132 including the program instructions. The program instructions may be executed by the processor 1122 of the electronic device 1100 to perform the above-described image processing method.
[0201] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a processor, and the computer program implements the steps of the above-mentioned image processing method when executed by the processor.
[0202] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.
[0203] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0204] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. An image processing method, characterized in that: include: For any specified threshold value among a plurality of specified threshold values, determining the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value, and the number of second pixels whose grayscale values are greater than the specified threshold value; Obtaining a difference corresponding to the specified threshold based on the first number of pixels and the second number of pixels; the difference indicating a difference between a first region and a second region in the image to be processed; Filtering the specified threshold corresponding to the maximum difference from the differences corresponding to the multiple specified thresholds as the target threshold; The target threshold is used as a basis for determining whether a pixel point in the image to be processed is an edge pixel point, and the edge pixel point is used to segment the first area and the second area.
2. The method according to claim 1, characterized in that Obtaining a difference corresponding to the specified threshold according to the first number of pixels and the second number of pixels includes: Determining a first pixel ratio of the first number of pixels to the total number of pixels in the image to be processed, and a second pixel ratio of the second number of pixels to the total number of pixels; Determining a first grayscale value sum of pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold, and a second grayscale value sum of pixels whose grayscale values are greater than the specified threshold; A degree of difference corresponding to the specified threshold is obtained according to the first pixel ratio, the second pixel ratio, the first grayscale value sum, and the second grayscale value sum.
3. The method according to claim 1, characterized in that The method further comprises: Inputting the image to be processed into a first model; For any pixel in the image to be processed, identifying, by the first model, a probability that the pixel is the edge pixel; When the probability that the pixel point is the edge pixel point is greater than the target threshold, it is determined that the pixel point is the edge pixel point.
4. The method according to claim 3, characterized in that The method further comprises: The second model is trained using multiple sets of training samples; each set of training samples includes a feature vector of a pixel sample extracted from an image sample by the first model and a classification result of the pixel sample; the classification result includes whether the pixel sample is an edge pixel or a non-edge pixel; The feature vector of the pixel point extracted by the first model from the image to be processed is input into the trained second model to determine whether the pixel point is the edge pixel point or the non-edge pixel point.
5. The method according to claim 1, wherein The method further comprises: Clustering the pixels in the image to be processed to obtain a plurality of third pixels and a plurality of fourth pixels; the plurality of third pixels are pixels in the first area, and the plurality of fourth pixels are pixels in the second area; Determine a first average value of the grayscale values of the plurality of third pixels and a second average value of the grayscale values of the plurality of fourth pixels; using a midpoint between the first average value and the second average value as an initial threshold value among the plurality of specified threshold values; The initial threshold value and a threshold value whose difference from the initial threshold value is smaller than a preset value are used as the multiple designated threshold values.
6. The method according to any one of claims 1 to 5, characterized in that Different images to be processed correspond to different target thresholds.
7. An image processing device, characterized in that: include: a determination module configured to determine, for any specified threshold value among a plurality of specified threshold values, the number of first pixels in the image to be processed whose grayscale values are less than or equal to the specified threshold value, and the number of second pixels whose grayscale values are greater than the specified threshold value; a difference module configured to obtain a difference corresponding to the specified threshold based on the first number of pixels and the second number of pixels; The difference degree indicates the difference between the first area and the second area in the image to be processed; a screening module configured to screen out the designated threshold corresponding to the maximum difference from the differences corresponding to the multiple designated thresholds as the target threshold; The target threshold is used as a basis for determining whether a pixel point in the image to be processed is an edge pixel point, and the edge pixel point is used to segment the first area and the second area.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.