Seedling growth dynamic character extraction method and system

Abstract

The invention discloses a seedling growth dynamic character extraction method and system, and the method comprises the steps: obtaining a dynamic growth video of a seedling process through delayed photography, carrying out the Gaussian fitting of a gray histogram of a video frame, obtaining a gray distribution interval of a soil region and an environment background, and carrying out the Gaussian fitting of the gray distribution interval; the method comprises the steps of dividing pixel points into three classes according to the relationship between gray values and gray distribution intervals of a soil area and an environment background, determining growth attention indexes of the pixel points according to the classes to which the pixel points belong and the distances between the gray values of the pixel points and boundaries of the distribution intervals, and determining the growth attention indexes of the pixel points according to the classes to which the pixel points belong and gradient features of the pixel points. Determining a dynamic space-time window of the pixel points; performing motion filtering on the dynamic growth video according to the dynamic space-time window of the pixel point and the growth attention index to obtain a filtered dynamic growth video; and analyzing the filtered dynamic growth video, and extracting seedling growth dynamic characters. According to the method, the accuracy and reliability of growth dynamic character extraction are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a method and system for extracting dynamic traits of seedling growth. Background Art

[0002] Through time-lapse photography technology, researchers can shoot seedlings at different stages of their growth at a high frequency, thus forming a full-scale, continuous dynamic image; this video can not only show the growth characteristics of plants under changes in light, temperature and moisture, but also effectively capture the instantaneous reactions caused by changes in the growth environment, which is of great significance for plant seedling research.

[0003] However, in the dynamic growth video of the seedling cultivation process shot by time-lapse photography, long-term changes (such as seedling growth) and short-term changes (such as light changes, camera shake, etc.) are mixed together. Short-term changes are manifested as shaking and trembling of the video, which will interfere with the analysis and understanding of the seedling growth status.

[0004] Therefore, it is necessary to perform motion filtering on the video frames, and identify and eliminate unnecessary motion noise in the video according to the spatiotemporal window of pixel points, so as to obtain the true dynamic characteristics of seedling plant growth; among them, the size and position of the spatiotemporal window determine the time dimension and spatial dimension considered in the filtering process, which in turn affects the recognition and removal effect of noise. Therefore, the setting of the spatiotemporal window has a significant impact on the motion filtering effect.

[0005] In the existing motion filtering, the same spatiotemporal window is set for all pixels, which affects the motion filtering effect and further affects the accuracy of the analysis and understanding of the growth status of the seedlings. Summary of the invention

[0006] To solve the technical problem that the same spatiotemporal window is set for all pixels in the above-mentioned existing motion filtering, which affects the motion filtering effect and further affects the accuracy of the analysis and understanding of the growth status of the seedlings, the present invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a method for extracting dynamic traits of seedling growth, comprising: obtaining a dynamic growth video of a seedling raising process by time-lapse photography; for video frames in the dynamic growth video, performing motion filtering on the dynamic growth video according to a dynamic spatiotemporal window and a growth focus index of a pixel point, to obtain a filtered dynamic growth video; analyzing the filtered dynamic growth video to extract dynamic traits of seedling growth; wherein, the method for obtaining the dynamic spatiotemporal window and the growth focus index of a pixel point is: performing Gaussian fitting on a grayscale histogram of a video frame, to obtain a grayscale distribution interval of a soil region and an environmental background; dividing the pixel points into three categories according to the relationship between the grayscale value and the grayscale distribution interval of the soil region and the environmental background, the obtained three categories respectively characterizing the pixel points in the environmental background, the soil region and the seedling region; determining the growth focus index of the pixel point according to the class to which the pixel point belongs and the distance between its grayscale value and the boundary of each distribution interval; determining the dynamic spatiotemporal window of the pixel point according to the class to which the pixel point belongs and the gradient characteristics of the pixel point, the dynamic spatiotemporal window being composed of local windows corresponding to the pixel point in multiple video frames on the time axis.

[0008] Preferably, the grayscale distribution interval of the soil area and the environmental background is obtained, including: for all grayscale values ​​whose frequencies are not equal to 0 in the grayscale histogram of the video frame, the minimum value and the maximum value thereof are recorded as the minimum grayscale value, respectively. and the maximum gray value ;right Gaussian fitting is performed on the grayscale values ​​within the range, and the mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will As the gray distribution interval of the soil area; Gaussian fitting is performed on the grayscale values ​​within the range to obtain the grayscale distribution interval of the environmental background. The mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will As the grayscale distribution interval of the environmental background; and is the preset grayscale boundary.

[0009] Preferably, the pixel points are divided into three categories, including: pixel points whose grayscale values ​​are within the grayscale distribution interval of the environmental background are divided into the first category; pixel points whose grayscale values ​​and local background grayscale values ​​are both within the grayscale distribution interval of the soil area are divided into the second category; pixel points whose grayscale values ​​are within the grayscale distribution interval of the soil area but the local background grayscale values ​​are not within the grayscale distribution interval of the soil area are divided into the third category; The pixels within are classified into the third category. is the right boundary of the grayscale distribution interval of the soil area, It is the left boundary of the grayscale distribution range of the environmental background.

[0010] Preferably, the process of obtaining the local background grayscale value of the pixel point includes: As the center, get a size equal to local area, is a positive integer greater than 2 and is an odd number; the size of the local area is continuously expanded until at least 8 pixels corresponding to the pixel point can be found in the local area. When the grayscale difference of two pixels is greater than 10, the expansion operation is stopped and the local area obtained at this time is used as the local background area of ​​the pixel; wherein the grayscale difference refers to the absolute value of the difference between the grayscale values ​​of two pixels.

[0011] Preferably, the determining of the growth concern index of the pixel includes: for the pixel in the first category, the growth concern index of the pixel is equal to 0.25; for the pixel in the second category, the calculation formula of the growth concern index of the pixel is: ; For the pixels in the third category, the calculation formula of the pixel growth attention index is: ; In the formula, Indicates the growth attention index of the pixel point, Indicates taking the minimum value, is an exponential function with a natural constant as base, Represents the local background gray value of the pixel, Represents the gray value of a pixel. is the grayscale distribution interval of the soil area, It is the left boundary of the grayscale distribution range of the environmental background.

[0012] Preferably, the process of obtaining the local windows corresponding to the pixel points in the multiple video frames on the time axis includes: for the pixel points in the first category or the second category, marking the coordinates of the pixel points as , then the local window corresponding to the pixel point in each video frame on the time axis is based on the coordinates in each video frame. As the pixel point of the third category, the target pixel point is taken as the target pixel point, and the video frame where the target pixel point is located is recorded as the video frame. , get the target pixel in the video frame The corresponding local window in , including: theoretical growth rate of seedlings and video frames With video frame The product of the time interval between the video frames The corresponding growth displacement; combined with the gray value of the target pixel and its local background gray value, according to the growth displacement and the gradient direction of the target pixel, determine the target pixel in the video frame According to the displacement vector, the target pixel is moved to obtain the video frame The target pixel point in the video frame The corresponding local window is a rectangular area of ​​7×7 centered on the positioning pixel, and it is required that two parallel sides in the rectangular area are parallel to the gradient direction of the target pixel, and the other two parallel sides are perpendicular to the gradient direction of the target pixel.

[0013] Preferably, the target pixel point is in the video frame The process of obtaining the displacement vector in includes: When the gray value of the target pixel is less than the gray value of its local background, the target pixel is in the video frame. The direction of the displacement vector in is opposite to the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the target pixel is in the video frame The direction of the displacement vector in is the same as the gradient direction of the target pixel; when When the gray value of the target pixel is less than the gray value of its local background, the target pixel is in the video frame. The direction of the displacement vector in is the same as the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the target pixel is in the video frame The direction of the displacement vector in is opposite to the gradient direction of the target pixel; and the modulus of the displacement vector is equal to the growth displacement.

[0014] Preferably, the motion filtering of the dynamic growing video is performed according to the dynamic spatiotemporal window and growth attention index of the pixel points, including: for the video frames of the dynamic growing video to be processed, the pixel points in the input video frame are smoothed through the dynamic spatiotemporal window of the pixel points, and it is assumed that each pixel point in the output video frame is obtained by displacing its corresponding input video frame in the displacement field, and when constructing the energy function, each item in the energy function is weighted by the growth attention index of the pixel point, and the displacement field is obtained by minimizing the energy function; the input video frame is displaced using the obtained displacement field to obtain the corresponding output video frame, and all the output video frames are combined to obtain the filtered dynamic growing video.

[0015] Preferably, the seedling growth dynamic traits refer to the growth significance of each period, and the specific acquisition includes: dividing all video frames into multiple periods, each period corresponds to video frames, is the shooting time interval between two adjacent video frames in the dynamic growth video, =6, Indicates rounding up; in all video frames corresponding to each period, the height of the seedlings in the first video frame and the last video is taken as the height at the beginning and end of each period, respectively, and the height difference between the end and the beginning of each period is taken as the growth of the seedlings in each period; the growth of the seedlings in each period is obtained, and the ratio of the average growth of all seedlings in the same period to the maximum growth of all seedlings in all periods is taken as the growth significance of the corresponding period.

[0016] In a second aspect, the present invention provides a system for extracting dynamic traits of seedling growth, comprising a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned method for extracting dynamic traits of seedling growth is implemented.

[0017] By adopting the above technical solution, the above-mentioned method for extracting dynamic characteristics of seedling growth is generated into a computer program and stored in a memory to be loaded and executed by a processor, so that a terminal device is made according to the memory and the processor for easy use.

[0018] The beneficial effects of the present invention are: The present invention sets different dynamic spatiotemporal windows for pixel points with different characteristics according to the characteristics of pixel points, can more accurately identify motion noise unrelated to seedling growth, reduce interference with video analysis, and at the same time, dynamically adjust the window to provide higher flexibility and accuracy, thereby improving the accuracy of dynamic traits of seedling growth obtained through dynamic growth video analysis of the seedling process; at the same time, the present invention calculates and sets growth attention indicators of pixel points according to the class to which the pixel points belong, sets different growth attention indicators for pixel points in characterizing environmental background, soil area and seedling area, and when motion filtering is performed on the dynamic growth video, weights each item in the energy function by the growth attention indicator of the pixel points, so that the energy field obtained by solving the energy function focuses on the key pixel points of seedling growth, that is, the pixel points in the seedling area, in a targeted manner, improves the clarity of the pixel points in the seedling area in the filtered dynamic growth video, thereby improving the accuracy and reliability of the extraction of dynamic growth traits. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below through the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1is a flow chart schematically showing a method for extracting dynamic traits of seedling growth in the present invention; Figure 2 is a schematic diagram schematically illustrating a space-time window; Figure 3 is a flowchart schematically illustrating setting a dynamic spatiotemporal window and a growth focus index for each pixel point in a video frame in step S2; Figure 4 is a schematic diagram schematically showing video frames of a dynamic growth video of a seedling raising process; Figure 5 It is schematically shown Figure 4 A schematic diagram of a grayscale histogram of a video frame shown; Figure 6 is a schematic diagram schematically illustrating a dynamic space-time window. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0022] The embodiment of the present invention discloses a method for extracting dynamic traits of seedling growth, referring to Figure 1 , including steps S1 to S3: S1. Obtain dynamic growth video of the seedling cultivation process through time-lapse photography.

[0023] Seedling cultivation is the process of growing seedlings, which means growing seedlings in a nursery, hotbed or greenhouse in preparation for transplanting them into the land for planting. Time-lapse photography of the seedling growth process is used to obtain dynamic growth videos of the seedling cultivation process, which can be used as visual data for extracting seedling growth traits.

[0024] The specific steps for obtaining dynamic growth videos of the seedling cultivation process through time-lapse photography are as follows: 1. Choose a camera with time-lapse photography function, including but not limited to SLR or mirrorless cameras that can manually set shutter speed and interval shooting; fix the camera with a tripod to avoid shaking during shooting.

[0025] 2. During the seedling raising process, in order to avoid the impact of changes in natural light and ensure that the light in the seedling growth environment is uniform and stable, use artificial lighting (such as LED growth lights); monitor and maintain appropriate temperature and humidity to provide a good growth environment for the seedlings so as to obtain a more natural growth process.

[0026] 3. Set the shooting time interval. In this embodiment, the shooting time interval is equal to 30 minutes; then the shooting time interval between each two adjacent frames of the dynamic growth video obtained subsequently is Equals 30 minutes.

[0027] 4. Set the frame rate, and obtain all images through time-lapse photography at the set frame rate to form a dynamic growth video of the seedling cultivation process, thereby forming a smooth dynamic effect; in this embodiment, the frame rate is equal to 30fps (frames per second), fps is the unit of frame rate, which means the number of frames displayed per second.

[0028] S2. For video frames in the dynamic growing video, a dynamic spatiotemporal window and a growth focus index of the pixel points are set; according to the dynamic spatiotemporal window and the growth focus index of the pixel points, motion filtering is performed on the dynamic growing video to obtain a filtered dynamic growing video.

[0029] It should be noted that in the dynamic growth video of the seedling cultivation process shot by time-lapse photography, long-term changes (such as seedling growth) and short-term changes (such as light changes, camera shake, etc.) are mixed together. Short-term changes are manifested as shaking and trembling in the video, which will interfere with the analysis and understanding of the growth status of the seedlings. Therefore, motion filtering technology is used to remove short-term changes in the video, such as light changes, slight camera shake, etc., to highlight the long-term growth changes of the seedlings.

[0030] The specific implementation steps of motion filtering are as follows: for the video frames of the dynamic growing video to be processed, a spatiotemporal window is constructed in time and space, the pixel points in the input video frame are smoothed through the spatiotemporal window, and it is assumed that each pixel point in the output video frame is obtained by displacing its corresponding input video frame in the displacement field, and the displacement field is obtained by minimizing the energy function; the input video frame is displaced using the obtained displacement field to obtain the corresponding output video frame, and all the output video frames are combined to obtain the filtered dynamic growing video; in this way, the action in the video frame of the dynamic growing video is decomposed into short-term changes and long-term changes, the short-term changes are filtered out, and the long-term changes are enhanced, so as to realize motion filtering processing of the dynamic growing video.

[0031] Among them, the energy function has three terms: the first term represents the fidelity, which means that the output video frame is obtained by displacing the input video frame, and the output video frame is required to be highly similar to the input video frame; the second term represents the time domain similarity, which means that the output video frame has local smoothness in the space-time window; the third term is the regularization term, which means that the displacement field of the pixel point has local smoothness.

[0032] The spatiotemporal window is composed of local windows of the same size corresponding to the pixel points in multiple video frames on the time axis. The schematic diagram of the spatiotemporal window is as follows: Figure 2 shown.

[0033] The motion filtering adopts the "motion denoising for time-lapse photography" method proposed by Michael Rubinstein et al. in the journal "IEEE Computer Vision and Pattern Recognition (CVPR), June 2011" in 2011; therefore, the motion filtering adopted in the present invention is a well-known technology and will not be described in detail here.

[0034] In motion filtering, the size and position of the spatiotemporal window determine the time and space dimensions considered in the filtering process, which in turn affects the recognition and removal of noise: if the spatiotemporal window is set too small, it may lead to over-filtering, that is, some normal dynamic changes are misjudged as noise, thereby losing important growth information; if the spatiotemporal window is set too large, smaller motion details may be screened out, resulting in insufficient filtering effect, and some unnecessary noise is still retained, which may also make the filtered image blurred; therefore, the setting of the spatiotemporal window has a significant impact on the motion filtering effect; in the existing motion filtering, the same spatiotemporal window is set for all pixels, which affects the motion filtering effect, and thus affects the accuracy of the analysis and understanding of the growth status of the seedlings.

[0035] Therefore, appropriately and dynamically setting the spatiotemporal window of pixels is the key to achieving effective motion filtering, which can help researchers better understand and analyze the growth process of seedlings.

[0036] In summary, the present invention sets different dynamic space-time windows for pixels with different characteristics according to the characteristics of the pixel points, and calculates the growth attention index of the pixel points. When motion filtering is performed on the dynamic growth video, the fixed space-time window is replaced by the dynamic space-time window. When constructing the energy function, each item in the energy function is weighted by the growth attention index of the pixel point. By dynamically adjusting the size of the space-time window according to the characteristics of the pixel points, motion noise that is not related to seedling growth can be more accurately identified, thereby reducing interference with video analysis. At the same time, dynamically adjusting the window can provide higher flexibility and accuracy, thereby improving the accuracy of growth analysis.

[0037] The flowchart of setting the dynamic spatiotemporal window and growth focus index for each pixel in the video frame in step S2 is shown in FIG. Figure 3 , including steps S201 to S204, specifically: S201, performing Gaussian fitting on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background.

[0038] It should be noted that in the dynamic video of seedling growth, it is crucial to accurately distinguish between the soil, environmental background and the seedlings themselves. By distinguishing the pixels in the seedling area from the pixels in the soil area and the environmental background, more attention can be paid to the dynamic growth information of the seedlings, such as the stretching of leaves and the increase in plant height.

[0039] The video frames of the dynamic growth video include seedlings, soil and environment. In the scene of seedling growth, the grayscale values ​​of pixels in the soil area and the environmental background are usually relatively concentrated and narrowly distributed due to factors such as their material and color. For example, the soil often appears darker gray or brown, and the corresponding grayscale values ​​are concentrated in a specific interval of the grayscale histogram close to the origin, while the environmental background has a brighter grayscale value, thereby forming a relatively independent distribution area of ​​the original origin on the grayscale histogram; moreover, the soil area is darker and the environmental background is brighter, that is, the grayscale value of the pixel in the soil area is smaller, while the grayscale value of the pixel in the environmental background is larger, and the pixel in the seedling area is between the two. Therefore, the present invention performs Gaussian fitting on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background.

[0040] Get the grayscale histogram of the video frame. The grayscale histogram can intuitively show the number distribution of pixels corresponding to each grayscale value in the video frame. The horizontal axis of the grayscale histogram is the grayscale value, and the vertical axis is the frequency of each grayscale value. The frequency refers to the gray value in the video frame equal to The number of times a pixel appears.

[0041] For example, a schematic diagram of a video frame of a dynamic growth video of a seedling raising process is as follows: Figure 4 As shown, Figure 4 The schematic diagram of the grayscale histogram of the video frame shown is as follows Figure 5 shown.

[0042] Gaussian fitting is performed on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background. The specific method is as follows: for all grayscale values ​​whose frequency is not equal to 0 in the grayscale histogram, the minimum and maximum values ​​are recorded as the minimum grayscale value, respectively. and the maximum gray value ;right Gaussian fitting is performed on the grayscale values ​​within the range, and the mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will As the gray distribution interval of the soil area; Gaussian fitting is performed on the grayscale values ​​within the range to obtain the grayscale distribution interval of the environmental background. The mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will The grayscale distribution interval as the environmental background.

[0043] in, and is the preset grayscale boundary. It is used to limit the grayscale distribution range of the soil area, and the preset grayscale boundary It is used to limit the grayscale distribution range of the environmental background area. The soil area is darker and the environmental background is brighter. That is to say, the grayscale value of the pixel in the soil area is smaller, while the grayscale value of the pixel in the environmental background is larger. Therefore, ; Preset grayscale boundary and The specific value can be set according to the actual application scenario and requirements, and the grayscale boundary is preset The value range is , preset grayscale boundary The value range is Therefore, this embodiment will preset the grayscale boundary Set to 40 to preset the grayscale boundary Set to 120.

[0044] S202, dividing the pixel points into three categories according to the relationship between the grayscale value and the grayscale distribution range of the soil area and the environmental background.

[0045] It should be noted that in the video frame, the color of the soil area is darker, the color of the seedling area is medium, and the color of the environmental background is lighter. Therefore, the grayscale values ​​of the pixels belonging to the soil area, the pixels belonging to the seedling area, and the pixels belonging to the environmental background increase successively. Based on this feature, the pixels belonging to the soil area, the pixels belonging to the seedling area, and the pixels belonging to the environmental background can be distinguished.

[0046] In particular, some pixels belonging to the seedling area have darker colors, resulting in smaller grayscale values ​​of these pixels, which fall within the grayscale distribution range of the soil area. In order to distinguish these pixels, the feature of the local background grayscale value of the pixel is introduced. If the local background grayscale value of the pixel also falls within the grayscale distribution range of the soil area, it can be determined that the pixel belongs to the soil area.

[0047] According to the relationship between the grayscale value and the grayscale distribution interval of the soil area and the environmental background, the pixels are divided into three categories, where the first category represents the pixels in the environmental background, the second category represents the pixels in the soil area, and the third category represents the pixels in the seedling area; the specific method is as follows: 1. If the gray value of the pixel is within the gray distribution range of the background It means that the pixel is in the background. Therefore, the gray value is in the gray distribution interval of the background. The pixels within are classified into the first category.

[0048] 2. If the grayscale value of the pixel is within the grayscale distribution range of the soil area, and the local background grayscale value of the pixel is also within the grayscale distribution range of the soil area, it means that the pixel is a pixel in the soil area. Therefore, the pixel whose grayscale value and local background grayscale value are both within the grayscale distribution range of the soil area are classified into the second category.

[0049] 3. If the grayscale value of the pixel is within the grayscale distribution range of the soil area, while the local background grayscale value of the pixel is not within the grayscale distribution range of the soil area, it means that the pixel is a pixel in the seedling area. Therefore, the pixel whose grayscale value is within the grayscale distribution range of the soil area while the local background grayscale value of the pixel is not within the grayscale distribution range of the soil area is classified into the third category.

[0050] 4. It is the grayscale interval between the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background, and it is also the grayscale distribution interval of most pixels in the seedling area. If the grayscale value of the pixel is It means that the pixel is in the seedling area, so the gray value is The pixels within are classified into the third category.

[0051] Among them, the pixel The method for obtaining the local background grayscale value is as follows: As the center, get a size equal to local area, is a positive integer greater than 2 and is an odd number; the size of the local area is continuously expanded until at least 8 pixels corresponding to the pixel point can be found in the local area. When the grayscale difference of two pixels is greater than 10, the expansion operation is stopped and the local area obtained at this time is used as the local background area of ​​the pixel; the grayscale difference refers to the absolute value of the difference between the grayscale values ​​of two pixels.

[0052] S203: Determine a growth focus index of the pixel point according to the class to which the pixel point belongs and the grayscale value of the pixel point.

[0053] It should be noted that the pixels close to the seedlings often contain key information about the growth of the seedlings, such as the edge of the leaves, the junction of the seedling stems and the soil, etc.; taking the edge of the leaves as an example, as the seedlings grow, the leaves gradually unfold and expand, and the pixels near their edges will show changes in grayscale values ​​or other characteristic values. When the leaves gradually stretch from a curled state to a flat state, the pixels near the edges change first, from a grayscale value similar to the inside of the leaves in the curled state to a grayscale value or color value that is significantly different from the soil background. These changes can keenly reflect the growth dynamics of the seedling leaves. Therefore, by setting the growth focus indicators of these pixels to be larger, these key information can be captured with higher priority, thereby more accurately monitoring and analyzing the growth of the seedlings.

[0054] 1. For the pixels in the environmental background, that is, the pixels in the first category, the growth concern index of the pixel is a fixed value, and the growth concern index of the pixel is equal to 0.25.

[0055] 2. For the pixels in the soil area, i.e. the pixels in the second category, the calculation formula of the growth concern index of the pixels is as follows: ; In the formula, Indicates the growth attention index of the pixel point, Indicates taking the minimum value, Represents the local background gray value of the pixel, is the grayscale distribution interval of the soil area, It is the right boundary of the grayscale distribution interval of the soil area.

[0056] in, is the grayscale distribution interval of the soil area, is the grayscale distribution interval of most pixels in the seedling area, is the key grayscale value to distinguish the soil area from the seedling area. Therefore, for the pixels in the soil area, that is, the pixels in the second category, the local background grayscale value of the pixels is With the key gray value The farther the distance, the The larger it is, the more likely that the pixel is a pixel near the seedling area. Therefore, compared with other pixels belonging to the soil area, its growth concern index should be set larger.

[0057] 3. For the pixels in the seedling area, that is, the pixels in the third category, the calculation formula for the growth concern index of the pixels is as follows: ; In the formula, Indicates the growth attention index of the pixel point, Represents the gray value of a pixel. represents an exponential function with a natural constant as base, is the grayscale distribution interval of the soil area The right boundary of is the grayscale distribution interval of the environmental background The left border of .

[0058] Among them, the gray distribution range of the soil area The right border And the grayscale distribution range of the environmental background Left border of , which constitutes the grayscale distribution range of most pixels in the seedling area ,and is the key gray value to distinguish the soil area from the seedling area. is the key grayscale value to distinguish the environmental background from the seedling area; the grayscale value of the stem part belonging to the seedling area is larger, while the grayscale value of the leaf part belonging to the seedling area is smaller; in the dynamic growth video of the seedling, more attention is paid to the growth of the stem than the growth of the leaf, because the growth of the stem is more stable than that of the leaf, and can better reflect the overall growth trend of the seedling; therefore, for the pixel point in the seedling area, that is, the pixel point in the third category, when the grayscale value of the pixel point is equal to the key grayscale value to distinguish the environmental background from the seedling area Distance Smaller, and the key gray value to distinguish the soil area from the seedling area Distance When it is larger, it means that the pixel point is more likely to belong to the stem part of the seedling area and less likely to belong to the leaf part of the seedling area, so the growth attention index of the pixel point is set to a larger value.

[0059] It should be noted that the present invention calculates and sets the growth attention index of the pixel point according to the class to which the pixel point belongs, and sets different growth attention indexes for the pixel points that characterize the environmental background, soil area and seedling area. When motion filtering is performed on the dynamic growth video, each item in the energy function is weighted by the growth attention index of the pixel point, so that the energy field obtained by solving the energy function focuses on the key pixel points of seedling growth, that is, the pixel points in the seedling area, thereby improving the clarity of the pixel points in the seedling area in the filtered dynamic growth video, thereby improving the accuracy and reliability of the extraction of dynamic growth traits.

[0060] S204: Determine a dynamic spatiotemporal window of the pixel point according to the class to which the pixel point belongs and the gradient characteristics of the pixel point.

[0061] The dynamic spatiotemporal window of a pixel point is composed of local windows corresponding to the pixel point in multiple video frames on the time axis, and the multiple video frames on the time axis include video frames , video frame , video frame , video frame , video frame , where the video frame is the video frame where the pixel is located, then the video frame and video frames is the video frame The two adjacent video frames before, video frame and video frames is the video frame The following two adjacent video frames; the schematic diagram of the dynamic spatiotemporal window of the pixel point is as follows Figure 6 shown.

[0062] Therefore, it is necessary to determine the local window corresponding to the pixel point in multiple video frames on the time axis. The specific method is as follows: 1. For the pixels in the background, i.e. the pixels in the first category, the coordinates of the pixels are marked as , then the local window corresponding to the pixel point in each video frame on the time axis is based on the coordinates in each video frame. A rectangular area with a size of 11×11 and centered at the pixel point.

[0063] 2. For the pixels in the soil area, i.e. the pixels in the second category, the coordinates of the pixels are marked as , then the local window corresponding to the pixel point in each video frame on the time axis is based on the coordinates in each video frame. A rectangular area with a size of 9×9 and centered at the pixel point.

[0064] It should be noted that, for the pixels in the seedling area, the positions of the corresponding pixels in the adjacent video frames change due to the growth of the seedlings, while the pixels in the environmental background and soil areas are basically not affected by the growth of the seedlings, and the positions of the corresponding pixels in the adjacent video frames do not change. Therefore, the positions of the local windows corresponding to the pixels in the environmental background and soil areas in each video frame on the time axis are fixed, while the positions of the local windows corresponding to the pixels in the seedling area in different video frames on the time axis are different, which needs to be determined based on the theoretical growth rate of the seedlings and the video frames. With video frame The time interval between the pixels and the calculated growth displacement is used to set the position of the local window corresponding to the pixel points in the seedling area in different video frames on the time axis.

[0065] 3. For the pixel points in the seedling area, that is, the pixel points in the third category, take them as target pixel points, and record the video frame where the target pixel points are located as video frame , get the target pixel in the video frame The corresponding local window method in is as follows, where: : 3.1. Calculate the gradient features of the target pixel using the Sobel operator. The gradient features include the gradient direction and gradient amplitude.

[0066] 3.2. According to the theoretical growth rate of seedlings and video frames With video frame The time interval between video frames The corresponding growth displacement.

[0067] Among them, the video frame With video frame The time interval is equal to , Indicates the shooting time interval between two adjacent video frames in the dynamic growth video; the theoretical growth rate refers to the length of the seedling growth per unit time, and the unit of measurement for the length of the seedling growth is the target pixel point. For example, the length of the seedling growth per unit time is 3 target pixels, so the theoretical growth rate is 3; then the video frame The corresponding growth displacement is equal to the product of the time interval between video frames and the theoretical growth speed.

[0068] 3.3. Combine the grayscale value of the target pixel and its local background grayscale value, determine the target pixel in the video frame according to the growth displacement and the gradient direction of the target pixel. The displacement vector in .

[0069] when Time, that is, video frame is the video frame In the subsequent video frames, the grayscale value of the target pixel and its local background grayscale value are determined: if the grayscale value of the target pixel is smaller than the local background grayscale value, the growth direction of the seedling area corresponding to the target pixel is opposite to the gradient direction of the target pixel. At this time, the seedling area corresponding to the target pixel is opposite to the gradient direction in the video frame. Find the target pixel point corresponding to the target pixel point in the video frame. Therefore, the target pixel point is The direction of the displacement vector in is opposite to the gradient direction of the target pixel, and the modulus of the displacement vector is equal to the growth displacement; if the gray value of the target pixel is greater than or equal to its local background gray value, the growth direction of the seedling area corresponding to the target pixel is the same as the gradient direction of the target pixel. At this time, along the gradient direction in the video frame Find the target pixel point corresponding to the target pixel point in the video frame. Therefore, the target pixel point is The direction of the displacement vector in is the same as the gradient direction of the target pixel, and the modulus of the displacement vector is equal to the growth displacement.

[0070] when Time, that is, video frame is the video frame In the previous video frame, the grayscale value of the target pixel and its local background grayscale value are judged: if the grayscale value of the target pixel is smaller than the local background grayscale value, the growth direction of the seedling area corresponding to the target pixel is the same as the gradient direction of the target pixel. At this time, along the gradient direction in the video frame Find the target pixel point corresponding to the target pixel point in the video frame. Therefore, the target pixel point is The direction of the displacement vector in is the same as the gradient direction of the target pixel, and the modulus of the displacement vector is equal to the growth displacement; if the gray value of the target pixel is greater than or equal to its local background gray value, the growth direction of the seedling area corresponding to the target pixel is opposite to the gradient direction of the target pixel. At this time, the growth direction of the seedling area in the video frame is opposite to the gradient direction. Find the target pixel point corresponding to the target pixel point in the video frame. Therefore, the target pixel point is The direction of the displacement vector in is opposite to the gradient direction of the target pixel, and the modulus of the displacement vector is equal to the growth displacement.

[0071] 3.4. According to the target pixel point in the video frame The displacement vector in moves the target pixel to obtain the video frame The positioning pixel in .

[0072] when When the target pixel is located in the video frame , the displacement vector does not exist, so the video frame The positioning pixel in is the target pixel.

[0073] when When the target pixel is located in the video frame Adjacent video frames, according to the target pixel point in the video frame The displacement vector in moves the target pixel point. The direction of movement is the direction of the displacement vector. The distance of movement is equal to the modulus length of the displacement vector. The video frame is obtained. The positioning pixel in .

[0074] 3.5. Based on video frames The pixel location in the video frame is obtained The corresponding local window in the video frame is The corresponding local window is based on each video frame A rectangular area with the positioning pixel in the center and a size of 7×7 is required, and two parallel sides in the rectangular area are parallel to the gradient direction of the target pixel, and the other two parallel sides are perpendicular to the gradient direction of the target pixel.

[0075] It should be noted that the present invention sets different dynamic space-time windows for pixels with different characteristics according to the characteristics of the pixel points, which can more accurately identify motion noise that is not related to seedling growth and reduce interference with video analysis. At the same time, the dynamic adjustment window can provide higher flexibility and accuracy, thereby improving the accuracy of the dynamic characteristics of seedling growth obtained through dynamic growth video analysis of the seedling process.

[0076] S3. Analyze the filtered dynamic growth video to extract the dynamic characteristics of seedling growth.

[0077] The dynamic characteristics of seedling growth refer to the growth significance at each stage, and the specific acquisition includes: Analyze the filtered dynamic growth video: divide all video frames into multiple periods, each period corresponds to video frames; in all video frames corresponding to each period, the heights of seedlings in the first video frame and the last video are taken as the heights at the beginning and end of each period, respectively, and the height difference between the end and the beginning of each period is taken as the growth of seedlings in each period; the growth of seedlings in each period is obtained, and the ratio of the average growth of all seedlings in the same period to the maximum growth of all seedlings in all periods is taken as the growth significance of the corresponding period.

[0078] is the shooting time interval between two adjacent video frames in the dynamic growth video, Indicates rounding up; in addition, the implementer can set The value of =6.

[0079] An embodiment of the present invention further discloses a system for extracting dynamic traits of seedling growth, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for extracting dynamic traits of seedling growth according to the present invention is implemented.

[0080] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

Claims

1. A method for extracting dynamic traits of seedling growth, characterized in that: include: The dynamic growth video of the seedling raising process is obtained by time-lapse photography; for the video frames in the dynamic growth video, the dynamic growth video is motion filtered according to the dynamic spatiotemporal window of the pixel points and the growth focus index to obtain the filtered dynamic growth video; the filtered dynamic growth video is analyzed to extract the dynamic growth characteristics of the seedling raising; Among them, the method for obtaining the dynamic spatiotemporal window and growth focus index of the pixel point is: Gaussian fitting is performed on the grayscale histogram of the video frame to obtain the grayscale distribution intervals of the soil area and the environmental background; the pixels are divided into three categories according to the relationship between the grayscale value and the grayscale distribution intervals of the soil area and the environmental background, and the three categories obtained respectively characterize the pixels in the environmental background, the soil area and the seedling area; the growth concern index of the pixel is determined according to the class to which the pixel belongs and the distance between its grayscale value and the boundary of each distribution interval; the dynamic space-time window of the pixel is determined according to the class to which the pixel belongs and the gradient characteristics of the pixel, and the dynamic space-time window is composed of local windows corresponding to the pixel in multiple video frames on the time axis.

2. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The step of obtaining the grayscale distribution interval of the soil area and the environmental background includes: For all grayscale values ​​whose frequencies are not equal to 0 in the grayscale histogram of the video frame, the minimum and maximum values ​​are recorded as the minimum grayscale value, respectively. and the maximum gray value ; right Gaussian fitting is performed on the grayscale values ​​within the range, and the mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will As the gray distribution interval of the soil area; Gaussian fitting is performed on the grayscale values ​​within the range to obtain the grayscale distribution interval of the environmental background. The mean and standard deviation of the obtained Gaussian fitting results are recorded as and ,Will As the grayscale distribution interval of the environmental background; and is the preset grayscale boundary.

3. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The pixels are divided into three categories, including: Pixels whose grayscale values ​​are within the grayscale distribution range of the environmental background are classified into the first category; Pixels whose grayscale values ​​and local background grayscale values ​​are both within the grayscale distribution range of the soil area are classified into the second category; Pixels whose grayscale values ​​are within the grayscale distribution interval of the soil area and whose local background grayscale values ​​are not within the grayscale distribution interval of the soil area are classified into the third category; The gray value in The pixels within are classified into the third category. is the right boundary of the grayscale distribution interval of the soil area, It is the left boundary of the grayscale distribution range of the environmental background.

4. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The process of obtaining the local background grayscale value of the pixel point includes: In pixels As the center, get a size equal to local area, is a positive integer greater than 2 and is an odd number; the size of the local area is continuously expanded until at least 8 pixels corresponding to the pixel point can be found in the local area. When the grayscale difference of the pixel is greater than 10, the expansion operation is stopped, and the local area obtained at this time is used as the local background area of ​​the pixel; The grayscale difference refers to the absolute value of the difference between the grayscale values ​​of two pixels.

5. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The step of determining the growth focus index of the pixel point includes: For pixels in the first category, the growth attention index of the pixel is equal to 0.25; For pixels in the second category, the calculation formula of the pixel growth attention index is: ； For pixels in the third category, the calculation formula of the pixel growth attention index is: ； In the formula, Indicates the growth attention index of the pixel point, Indicates taking the minimum value, is an exponential function with a natural constant as base, Represents the local background gray value of the pixel, Represents the gray value of a pixel. is the grayscale distribution interval of the soil area, It is the left boundary of the grayscale distribution range of the environmental background.

6. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The process of obtaining the local window corresponding to the pixel point in multiple video frames on the time axis includes: For pixels in the first or second category, the coordinates of the pixels are marked as , then the local window corresponding to the pixel point in each video frame on the time axis is based on the coordinates in each video frame. A rectangular area with the pixel point as the center and the size equal to 11×11 or 9×9; For the pixel points in the third category, take them as the target pixel points, and record the video frame where the target pixel points are located as the video frame , get the target pixel in the video frame The corresponding local window in , including: theoretical growth rate of seedlings and video frames With video frame The product of the time interval between the video frames The corresponding growth displacement; combined with the gray value of the target pixel and its local background gray value, according to the growth displacement and the gradient direction of the target pixel, determine the target pixel in the video frame According to the displacement vector, the target pixel is moved to obtain the video frame The target pixel point in the video frame The corresponding local window is a rectangular area of ​​7×7 centered on the positioning pixel, and it is required that two parallel sides in the rectangular area are parallel to the gradient direction of the target pixel, and the other two parallel sides are perpendicular to the gradient direction of the target pixel.

7. A method for extracting dynamic traits of seedling growth according to claim 6, characterized in that: The target pixel point is in the video frame The process of obtaining the displacement vector in includes: when When the gray value of the target pixel is less than the gray value of its local background, the target pixel is in the video frame. The direction of the displacement vector in is opposite to the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the target pixel is in the video frame The direction of the displacement vector in is the same as the gradient direction of the target pixel; when When the gray value of the target pixel is less than the gray value of its local background, the target pixel is in the video frame. The direction of the displacement vector in is the same as the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the target pixel is in the video frame The direction of the displacement vector in is opposite to the gradient direction of the target pixel; And the modulus of the displacement vector is equal to the growth displacement.

8. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The method of performing motion filtering on the dynamic growth video according to the dynamic spatiotemporal window of the pixel points and the growth attention index includes: For the video frames of the dynamic growing video to be processed, the pixels in the input video frames are smoothed through a dynamic spatiotemporal window of the pixels, and it is assumed that each pixel in the output video frame is obtained by displacing its corresponding input video frame in the displacement field. When constructing the energy function, each item in the energy function is weighted by the growth attention index of the pixel points, and the displacement field is obtained by minimizing the energy function; the input video frame is displaced using the obtained displacement field to obtain the corresponding output video frame, and all the output video frames are combined to obtain the filtered dynamic growing video.

9. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that: The seedling growth dynamic traits refer to the growth significance of each period, and the specific acquisition includes: Divide all video frames into multiple periods, each period corresponds to video frames, is the shooting time interval between two adjacent video frames in the dynamic growth video, =6, Indicates rounding up; In all video frames corresponding to each period, the height of the seedlings in the first video frame and the last video is used as the height at the beginning and end of each period, respectively, and the height difference between the end and the beginning of each period is used as the growth of the seedlings in each period; the growth of the seedlings in each period is obtained, and the ratio of the average growth of all seedlings in the same period to the maximum growth of all seedlings in all periods is used as the growth significance of the corresponding period.

10. A seedling growth dynamic trait extraction system, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for extracting dynamic traits of seedling growth according to any one of claims 1-9 is implemented.

Images