A fruit and vegetable video foreground detection method based on adaptive updating parameters

Abstract

The application discloses a fruit and vegetable video foreground detection method based on adaptive updating parameters, and the method is characterized in that: through an iterative process, parameters of different hypergeometric distributions for different frames are given, and a normal equation set and a matrix form are given by making the first derivative of the weight of all hypergeometric distributions in a frame equal to zero through a square error weighted sum function, so as to obtain a matrix expression of the weight of different hypergeometric distributions in the frame, wherein the parameters of the hypergeometric distribution can be adaptively updated after a certain frame in an interval period, and then the size of the weight corresponding to different hypergeometric distributions is selected in turn for a certain pixel point in the frame in the fruit and vegetable video sequence, so that fruit and vegetable video foreground detection is completed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of postharvest processing of fruits and vegetables, and particularly relates to giving parameters of different hypergeometric distributions for different frames through an iterative process, and giving normal equations and matrix forms for the first derivative of the weights of all hypergeometric distributions in a frame being zero through an error square weighted sum function to obtain a matrix expression of the weights of different hypergeometric distributions in the frame, wherein the parameters of the hypergeometric distributions can be adaptively updated after a certain frame in an interval period, so that for a certain pixel point in the frame in a fruit and vegetable video sequence, the corresponding hypergeometric distribution is selected in turn according to the size of the corresponding weight of different hypergeometric distributions at the frame, to complete fruit and vegetable video foreground detection. BACKGROUND

[0002] As a basic step in computer vision and video processing, foreground detection is widely used and has become one of the hot research topics in the field of computer vision, and has good research significance and application value. Foreground detection is a key step in video processing in the postharvest processing of fruits and vegetables, and the quality of the detection result directly relates to the advantages and disadvantages and practicality of subsequent analysis and processing. Generally, after inputting a fruit and vegetable video, foreground detection technology can be used to pre-process the image. Foreground detection is a process of identifying a region of interest or identifying a moving target (foreground) and a static part (background) in a video. Foreground detection needs to identify a region of interest or identify a moving fruit and vegetable target in a video, and due to the quality of the video and the complexity of the real scene, it is difficult to achieve detection. SUMMARY

[0003] The purpose of the present application is to set a premise that if each pixel in a fruit and vegetable video sequence can be represented by a color parameter and the observed value of the color parameter of a certain pixel point in a frame can be represented by a hypergeometric distribution constructed by a weighted sum of several hypergeometric distributions, through an iterative process, parameters of different hypergeometric distributions for different frames are given, and normal equations and matrix forms for the first derivative of the weights of all hypergeometric distributions in a frame being zero are given through an error square weighted sum function to obtain a matrix expression of the weights of different hypergeometric distributions in the frame, so that for a certain pixel point in the frame in a fruit and vegetable video sequence, the corresponding hypergeometric distribution is selected in turn according to the size of the corresponding weight of different hypergeometric distributions at the frame, and if the probability corresponding to a certain hypergeometric distribution is greater than or equal to a foreground probability threshold, it can be considered to belong to the hypergeometric distribution in the hypergeometric distribution foreground model.

[0004] To achieve the above purpose, the technical solution of the present application is as follows:

[0005] A fruit and vegetable video foreground detection method based on adaptive updating of parameters, the method comprising the following steps:

[0006] Step 1: If each pixel in the fruit and vegetable video sequence can be represented by a color parameter, and the color parameter observation value of a pixel point in a frame can be represented by a hypergeometric distribution foreground model constructed by a weighted sum of several hypergeometric distributions, and the parameters of the hypergeometric distribution can be adaptively updated after a certain frame in the interval period, then the predetermined mean of a certain hypergeometric distribution in a frame can be set to represent the average value of the color parameter corresponding to any pixel in the pixel point sample set in any frame within the upper and lower limits of the color parameter set for the hypergeometric distribution, and at the same time, the predetermined variance of the hypergeometric distribution in the frame can be set to represent the average value of the square of the value within the range of the upper and lower limits of the color parameter set for the hypergeometric distribution minus the square of the corresponding predetermined mean;

[0007] Step 2: Before the start of the iteration process, first compare the predetermined mean and predetermined variance of a certain hypergeometric distribution in a frame with the predetermined mean and predetermined variance before the interval period, if the absolute value of the deviation is within the mean deviation tolerance and the variance deviation tolerance respectively, then use the parameters before the interval period, otherwise enter the iteration process; In the iteration process, the parameters of the hypergeometric distribution in the frame can be adjusted by plus one or minus one under the premise of meeting the requirements of the hypergeometric distribution, and then the predetermined mean and predetermined variance corresponding to the current iteration number are compared with the mean and variance of the hypergeometric distribution obtained from the empirical formula, if the absolute value of the deviation is within the mean deviation tolerance and the variance deviation tolerance, then end the iteration process, otherwise re-enter the iteration process; At the end of the iteration process, the parameters of the hypergeometric distribution in the frame corresponding to the last iteration number are output;

[0008] Step 3: The error square weighted sum function is established by the weighted sum of the square of the error between the value probability of the color parameter observation value of any pixel in the pixel point sample set in any frame and the modeling value of the hypergeometric distribution, and the reference value ratio of the pixel in the frame; The second derivative of the weight of a certain hypergeometric distribution in a frame with respect to the error square weighted sum function is always positive, so when the first derivative of the weight of a certain hypergeometric distribution in a frame with respect to the error square weighted sum function is zero, it is known that the error square weighted sum function is at a minimum value at this time; Considering that in the case of time consistency, different hypergeometric distributions all have the first derivative of the weight of the above hypergeometric distribution in the frame with respect to the error square weighted sum function being zero, so for all hypergeometric distributions, when they are in the same frame scene, they can be rewritten as a certain normal equation group and matrix form, so that the matrix representation of the weight of all hypergeometric distributions in the frame is known;

[0009] Step 4: arranging the weights corresponding to different hypergeometric distributions in descending order at a certain frame, then selecting the hypergeometric distribution corresponding to the weight in descending order for a pixel point in the fruit and vegetable video sequence at the frame, and if the probability corresponding to the hypergeometric distribution is greater than or equal to the foreground probability threshold, the pixel point is considered to belong to the hypergeometric distribution in the hypergeometric distribution foreground model.

[0010] Preferably, the specific method of step 1 is:

[0011] If each pixel in the fruit and vegetable video sequence can be represented by a color parameter, and the color parameter observation value X g,t of the pixel point g at the t-th frame is t , the probability of k h,t can be represented by a hypergeometric distribution foreground model, where the parameters M h,t , N h,t and n t of the hypergeometric distribution h can be adaptively updated after an interval period T frames:

[0012]

[0013] where k h,t is the expected value of the pixel point g at the t-th frame, h is the hypergeometric distribution number, Ξ is the set of hypergeometric distribution numbers, the number of elements in Ξ can be set as H, M h,t , N h,t and n h,t are the parameters of the hypergeometric distribution h at the t-th frame, all of which are positive integers, and M h,t ≤ N h,t , n h,t ≤ M h,t , μ l is the weight of the hypergeometric distribution h at the t-th frame;

[0014] The predetermined mean value of the hypergeometric distribution h at the t-th frame can be represented as:

[0015]

[0016] where l is floor(t, T), the floor function is the floor function, Ψ l is the pixel point sample set in the range from the (l-L)T-th frame to the lT-th frame, L is the interval period coefficient, if l-L<0, the reference fruit and vegetable video sequence outside the fruit and vegetable video sequence can be considered to be added to the Ψ i,j , sum is the counting function, U(n) is the step function, x l is the color parameter corresponding to the pixel point i at the j-th frame in the Ψ h , B h and Aupper and lower limits of the color parameters of hypergeometric distribution h, respectively;

[0017] setting the predetermined variance of hypergeometric distribution h at the t-th frame may be expressed as:

[0018]

[0019] Preferably, the specific method of step 2 is:

[0020] Step 2.1: comparing the predetermined mean value of hypergeometric distribution h at the t-th frame and the predetermined variance with the predetermined mean value and the predetermined variance before the T-th frame, if ≤ δ E and , the parameters M h,t , N h,t and n h,t of hypergeometric distribution h at the t-th frame are set as M h,(l-1)T , N h,(l-1)T and n h,(l-1)T , respectively, where δ E and δ D are the mean deviation tolerance and the variance deviation tolerance, respectively, otherwise, go to step 2.2;

[0021] Step 2.2: setting the iteration number d as 0, and taking M h,(l-1)T , N h,(l-1)T and n h,(l-1)T as the initial state M h,t (d=0), N h,t (d=0) and n h,t (d=0) of the parameters M h,t , N h,t and n h,t of hypergeometric distribution h at the t-th frame, respectively;

[0022] Step 2.3: replacing d with d+1, and adjusting M h,t (d+1) = M h,t (d) ± 1, N h,t (d+1) = N h,t (d) ± 1 and n h,t (d+1) = n h,t (d) ± 1 under the premise of ensuring M h,t (d+1) ≤ N h,t (d+1), n h,t (d+1) ≤ M h,t (d+1);

[0023] Step 2.4: if If the condition is met, proceed to step 2.5; otherwise, proceed to step 2.3.

[0024] Step 2.5: Output M h,t (d+1), N h,t (d+1) and n h,t (d+1) is M in step 3 h,t N h,t and n h,t Use it;

[0025] Preferably, the specific method for step 3 is as follows:

[0026] Step 3.1: For the pixel sample set Ψ in the range from frame (lL)T to frame lT... l The error squared weighted sum function F(μ) for modeling numerical values ​​of medium probability and hypergeometric distribution h,t This can be represented as:

[0027]

[0028] Among them, Y i.j For the Ψ l The color parameter observation value of pixel i in frame j is x. i.j The probability, τ i,j For the Ψ l The reference value ratio of pixel i in frame j;

[0029] Step 3.2: From the F(μ) h,t ) for the μ h,t Taking the first and second derivatives, we get:

[0030]

[0031] Step 3.3: From It can be known that it is always positive, and it can be known that when When F(μ) h,t ) is the minimum value, at which point by We can obtain:

[0032]

[0033] set up We can obtain:

[0034]

[0035] Considering that different h exist under the condition that t is consistent When modeling H hypergeometric distributions all within the scene of frame t, the equations can be rewritten as follows:

[0036]

[0037] Step 3.4: The normal equation system described in Step 3.3 can be represented in the following matrix form:

[0038]

[0039] set up And order

[0040] Therefore, the μ h,t matrix representation

[0041] Preferably, the specific method for step 4 is as follows:

[0042] The μ described in step 3.4 h,t The matrix representation gives the weights μ corresponding to different hypergeometric distributions h at frame t. h,t Arranged from largest to smallest, the pixel g at frame t in the fruit and vegetable video sequence is weighted by μ. h,t The size is selected sequentially from the hypergeometric distribution h, and the color parameter observation value of pixel g in the t-th frame is set to X. g,t If the probability corresponding to the hypergeometric distribution h is If the probability is greater than or equal to the foreground probability threshold λ, it can be considered to belong to the hypergeometric distribution h in the hypergeometric distribution foreground model. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the steps of foreground detection in fruit and vegetable videos based on adaptive parameter updates.

[0044] Figure 2 This describes the specific flow of the iterative process in step 2;

[0045] Figure 3 These are the specific steps of the method in step 3;

[0046] Figure 4 The weighting states corresponding to different hypergeometric distributions. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the following description, in conjunction with the accompanying drawings, illustrates a specific implementation of the fruit and vegetable video foreground detection method based on adaptive parameter updates applied to navel orange videos.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a method for foreground detection in fruit and vegetable videos based on adaptive parameter updates, comprising the following steps:

[0049] Step 1: For the navel orange video with resolution of 704x576, convert the images in the video sequence to YCbCr color space, and then perform the hypergeometric distribution foreground model on each pixel in the image in the color space, the color parameter observation value X g,t is the probability of k t , where the parameters M h,t , N h,t and n h,t of the hypergeometric distribution h can be adaptively updated after an interval period T = 30 frames:

[0050]

[0051] where k t is the expected value of the pixel g in the YCbCr color space corresponding to the color parameter at the t-th frame, assuming that the pixel value of the image is 0 to 255, h is the hypergeometric distribution number, Ξ is the set of hypergeometric distribution numbers, the number of elements in Ξ is set to 7, M h,t , N h,t and n h,t are the parameters of the hypergeometric distribution h at the t-th frame, all of which are positive integers, and M h,t ≤ N h,t , n h,t ≤ M h,t , μ h,t is the weight of the hypergeometric distribution h at the t-th frame;

[0052] The predetermined mean value of the hypergeometric distribution h at the t-th frame can be represented as:

[0053]

[0054] The predetermined variance of the hypergeometric distribution h at the t-th frame can be represented as:

[0055]

[0056] Step 2 includes 5 steps as shown in Figure 2 :

[0057] Step 2.1: Compare the predetermined mean value and the predetermined variance of the hypergeometric distribution h at the t-th frame with the predetermined mean value and the predetermined variance before T frames, if and , then set the parameters M h,t , N h,t and n h,t of the hypergeometric distribution h at the t-th frame.M h,(l-1)T , N h,(l-1)T and n h,(l-1)T , respectively, where δ E and δ D are mean deviation tolerance and variance deviation tolerance, respectively, both of which can be set as 0.001, otherwise go to step 2.2;

[0058] Step 2.2: Set iteration number d as 0, and M h,(l-1)T , N h,(l-1)T and n h,(l-1)T as initial state M h,t (d=0), N h,t (d=0) and n h,t (d=0) of parameters M h,t , N h,t and n h,t of hypergeometric distribution h in t-th frame, respectively;

[0059] Step 2.3: Replace d with d+1, and adjust M h,t (d+1)=M h,t (d)±1, N h,t (d+1)=N h,t (d)±1 and n h,t (d+1)=n h,t (d)±1 under the premise of ensuring M h,t (d+1)≤N h,t (d+1), n h,t (d+1)≤M h,t (d+1);

[0060]

[0061] Go to step 2.5, otherwise go to step 2.3;

[0062] Step 2.5: Output M h,t (d+1), N h,t (d+1) and n h,t (d+1) as M h,t , N h,t and n h,t in step 3 for use;

[0063] Step 3 includes 4 steps as shown in Figure 3 :

[0064] Step 3.1: For pixel point sample set Ψ l in the range of (l-L)th frame to lth frame, error square weighted sum function F(μ h,t ) of probability and hypergeometric distribution modeling value can be expressed as:

[0065]

[0066] Step 3.2: The first derivative and the second derivative of F(μ h,t ) with respect to μ h,t are calculated as follows:

[0067]

[0068] Step 3.3: It is known that F(μ is always positive, and F(μ h,t ) is a minimum value when μ , and the following equation is obtained:

[0069]

[0070] Let μ , and the following equation is obtained:

[0071]

[0072] Considering that different h exist when t is consistent, and , the following normal equations are obtained for H hypergeometric distribution modeling when the scene is in the t-th frame:

[0073]

[0074] Step 3.4: The normal equations in step 3.3 can be expressed in the following matrix form:

[0075]

[0076] Let , and let

[0077] Therefore, the matrix expression of μ h,t is

[0078] Step 4: The matrix expression of μ h,t is given to the t-th frame, and the weights μ h,t corresponding to different hypergeometric distributions h are arranged in descending order. Then, the pixel point g in the t-th frame of the fruit and vegetable video sequence is selected by the size of the weight μ h,t in turn. In the calculation, the hypergeometric distribution probability can be given by the Stirling formula, and in practical applications, when N≥1000, considering that the limit of the hypergeometric distribution is the binomial distribution, and the limit of the binomial distribution is the normal distribution, the hypergeometric distribution can approach the normal distribution under certain conditions when N is large. Set the color parameter observation value X of the pixel point g in the t-th frame​g,t If μ is 0 and the foreground probability threshold λ is 0.5, then μ is set to... h,t The distribution is as follows Figure 4 As shown, when μ is selected first 4,120 When, the corresponding parameter M h,t N h,t and n h,t The values ​​are 8, 38, and 6 respectively. The corresponding hypergeometric distribution probability is 0.215, which does not satisfy the condition of being greater than or equal to the foreground probability threshold. Therefore, μ is selected sequentially... 1,120 The corresponding parameter M h,t N h,t and n h,t The values ​​are 6, 69, and 5 respectively. The corresponding hypergeometric distribution probability is 0.625. If the probability is greater than or equal to the foreground probability threshold, it can be considered to belong to the first type of hypergeometric distribution in the hypergeometric distribution foreground model.

[0079] Any parts not explicitly stated in this embodiment can be implemented using existing technologies.

[0080] For those skilled in the art, any changes, modifications, substitutions, and variations made to the embodiments without departing from the principles and spirit of the present invention, based on the teachings of the present invention, still fall within the protection scope of the present invention.

Claims

1. A method for foreground detection in fruit and vegetable videos based on adaptive parameter updating, characterized in that, The method includes the following steps: Step 1: Suppose that each pixel in the fruit and vegetable video sequence is represented by a color parameter and the color parameter is used in the first step... t The color parameter observations of any pixel in a frame are represented by a hypergeometric distribution foreground model constructed using a weighted hypergeometric distribution. The predetermined mean of this hypergeometric distribution in frame t represents the average value of the color parameter corresponding to any pixel in the pixel sample set at any frame within the range between the upper and lower limits set for the color parameters of this hypergeometric distribution. Simultaneously, the predetermined variance of this hypergeometric distribution in frame t represents the average of the squares of the values ​​of the color parameter corresponding to any pixel in the pixel sample set at any frame within the range between the upper and lower limits set for the color parameters of this hypergeometric distribution, minus the square of the corresponding predetermined mean. The parameters of the hypergeometric distribution are defined at intervals... T Adaptive updates are performed after each frame. Step 2: Before the iteration process begins, the predetermined mean and variance of the hypergeometric distribution in frame t described in Step 1 are compared with the predetermined mean and variance before the interval period. If the absolute value of the deviation is within the tolerance of mean deviation and variance deviation, respectively, the parameters before the interval period are used; otherwise, the iteration process begins. During the iteration process, the parameters of the hypergeometric distribution in frame t are adjusted by adding or subtracting one while ensuring the requirements of the hypergeometric distribution parameters. Then, the predetermined mean and variance corresponding to the current iteration number are compared with the mean and variance of the hypergeometric distribution obtained from empirical formulas. If the absolute value of the deviation is within the tolerance of mean deviation and variance deviation, the iteration process ends; otherwise, the iteration process restarts. At the end of the iteration process, the parameters of the hypergeometric distribution in the frame corresponding to the last iteration number are output. Step 3: Establish an error squared weighted sum function by weighting the square of the error between the probability of the observed color parameter value of any pixel in any frame and the value modeled by the hypergeometric distribution in the pixel sample set, and the reference value ratio of that pixel in that frame. The second derivative of the error squared weighted sum function with respect to the weight of the hypergeometric distribution in step 2 at frame t is always positive. Therefore, when the first derivative of the error squared weighted sum function with respect to the weight of the hypergeometric distribution at frame t is zero, the error squared weighted sum function is at its minimum. Considering that different hypergeometric distributions have a zero first derivative of the error squared weighted sum function with respect to the weight of the hypergeometric distribution at frame t under the same time conditions, the equations for all hypergeometric distributions in the same scene at frame t can be rewritten as a normal system of equations and a matrix form, thus revealing the matrix representation of the weights of all hypergeometric distributions at frame t. Step 4: Arrange the weights corresponding to different hypergeometric distributions in descending order at frame t. Then, for any pixel in frame t of the fruit and vegetable video sequence, select the corresponding hypergeometric distribution in order of the weights corresponding to different hypergeometric distributions in that frame. If the probability corresponding to the hypergeometric distribution is greater than or equal to the foreground probability threshold, it is considered to belong to the hypergeometric distribution in the hypergeometric distribution foreground model.

2. The method for foreground detection of fruit and vegetable videos based on adaptive parameter update according to claim 1, characterized in that, The specific method for step 1 is as follows: If each pixel in the fruit and vegetable video sequence is represented by a color parameter and in the th... Frame pixels Color parameter observations for The probability is represented by the hypergeometric distribution foreground model: in, For pixels In the Expected value per frame. Numbered according to the hypergeometric distribution. For the hypergeometric distribution number set, the The number of elements in the middle is set to , , and They are hypergeometric distributions In the The parameters of the frame all take positive integer values, and , Hypergeometric distribution In the Frame weights, the first Frame-time hypergeometric distribution parameters , and During the interval period Adaptive updates are performed after each frame, and a hypergeometric distribution is set. In the Predicted average of frames express: in, for , floor The function is to round down. For the first Frame to the A set of pixel samples within a frame. For the interval period coefficient, if At that time, a reference fruit and vegetable video sequence other than the above-mentioned fruit and vegetable video sequence will be added to the above-mentioned fruit and vegetable video sequence. middle, For counting functions, It is a step function. For the medium pixel In the The color parameters corresponding to the frame. and They are hypergeometric distributions The upper and lower limits of the color parameters are set; Define hypergeometric distribution In the Predetermined variance of frames express: 。 3. The method for foreground detection in fruit and vegetable videos based on adaptive parameter update according to claim 1, characterized in that, The specific method for step 2 is as follows: Step 2.1: Apply hypergeometric distribution In the Predicted average of frames and predetermined variance and The pre-defined average value before the frame and predetermined variance If a comparison is made, and Then set at the Hypergeometric distribution of frames parameters , and They are respectively , and ,in and These are the tolerance for mean deviation and the tolerance for variance deviation, respectively; otherwise, proceed to step 2.

2. Step 2.2: Set the number of iterations If it is 0, then , and As respectively in the first Hypergeometric distribution of frames parameters , and initial state , and ; Step 2.3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require replace In order to ensure Adjustment under the premise , and ; Step 2.4: If and If yes, proceed to step 2.5; otherwise, proceed to step 2.

3. Step 2.5: Output , and As in step 3 , and Use it.

4. The method for foreground detection in fruit and vegetable videos based on adaptive parameter update according to claim 1, characterized in that, The specific method for step 3 is as follows: Step 3.1: For the first Frame to the Pixel sample set within the frame range Weighted sum of squared errors function for numerical modeling of medium probability and hypergeometric distributions Represented as: in, For the medium pixel In the The observed value of the color parameter at frame time is The probability, For the medium pixel In the Frame-time reference value ratio; Step 3.2: From the above Regarding the Taking the first and second derivatives, we get: Step 3.3: From Knowing its constant correctness, one can know what should be done. hour It is the minimum value, at which point it is determined by... We can obtain: set up We can obtain: Considering Different under the same conditions All exist For H The hypergeometric distribution modeling is in the same period of the first In the frame scene, it can be rewritten as the following normal equation set 1: Step 3.4: Represent the normal equation system described in Step 3.3 in the following matrix form 1: set up And order ; Therefore, it can be concluded that the above matrix representation .

5. The method for foreground detection in fruit and vegetable videos based on adaptive parameter update according to claim 1, characterized in that, The specific method for step 4 is as follows: The steps described in step 3.4 The matrix representation of the first is given Frame time for different hypergeometric distributions Corresponding weights Arranged from largest to smallest, then for the [number]th [item] in the fruit and vegetable video sequence... Pixels per frame By weight The size is selected sequentially from the hypergeometric distribution. , set at Frame pixels The observed color parameter values ​​are If it corresponds to the hypergeometric distribution probability Greater than or equal to the foreground probability threshold Then it is considered to belong to the hypergeometric distribution in the hypergeometric distribution foreground model. .

Images