A method and system for extracting dynamic traits of seedling growth

Abstract

The present invention discloses a method and system for extracting dynamic growth traits of seedlings, including: obtaining a dynamic growth video of the seedling raising process through time-lapse photography, performing Gaussian fitting on the grayscale histogram of video frames to obtain the grayscale distribution intervals of the soil area and the environmental background, dividing pixel points into three categories according to the relationship between the grayscale value and the grayscale distribution intervals of the soil area and the environmental background, determining the growth attention index of pixel points according to the category to which the pixel points belong and the distance between their grayscale values and the boundaries of each distribution interval, and determining the dynamic spatio-temporal window of pixel points according to the category to which the pixel points belong and the gradient characteristics of the pixel points; performing motion filtering on the dynamic growth video according to the dynamic spatio-temporal window and the growth attention index of pixel points to obtain a filtered dynamic growth video; analyzing the filtered dynamic growth video to extract the dynamic growth traits of seedlings. The present invention improves the accuracy and reliability of extracting dynamic growth traits.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to a method and system for extracting dynamic traits of seedling growth. Background Art

[0002] Through time-lapse photography technology, researchers can take high-frequency pictures of different stages of seedlings during growth, thus forming an all-round and continuous dynamic image; such videos can not only show the growth characteristics of plants under changes in light, temperature and moisture, but also effectively capture the instantaneous reactions caused by changes in the growth environment, which is of great significance for plant seedling research.

[0003] However, in the dynamic growth videos of the seedling cultivation process taken by time-lapse photography, long-term changes (such as seedling growth) and short-term changes (such as light changes, camera jitter, etc.) are mixed together, and the short-term changes are manifested as the tremors and vibrations of the video, which will interfere with the analysis and understanding of the growth state of seedlings.

[0004] Therefore, it is necessary to perform motion filtering on video frames, identify and eliminate unnecessary motion noise in the video according to the spatio-temporal window of pixel points, so as to obtain the true dynamic characteristics of the growth of seedling plants; among them, the size and position of the spatio-temporal window determine the time dimension and space dimension considered in the filtering process, and thus affect the recognition and removal effect of noise. Therefore, the setting of the spatio-temporal window has a significant impact on the motion filtering effect.

[0005] In the existing motion filtering, the same spatio-temporal window is set for all pixel points, which affects the motion filtering effect and further affects the accuracy of the analysis and understanding of the growth state of seedlings. Summary of the Invention

[0006] To solve the above technical problem that in the existing motion filtering, the same spatio-temporal window is set for all pixel points, which affects the motion filtering effect and further affects the accuracy of the analysis and understanding of the growth state of seedlings, the present invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a method for extracting dynamic growth traits of seedlings, including: obtaining a dynamic growth video of the seedling raising process through time-lapse photography; for the video frames in the dynamic growth video, performing motion filtering on the dynamic growth video according to the dynamic spatio-temporal window of pixel points and growth attention indicators to obtain a filtered dynamic growth video; analyzing the filtered dynamic growth video to extract the dynamic growth traits of seedlings; wherein, the method for obtaining the dynamic spatio-temporal window of pixel points and growth attention indicators is: performing Gaussian fitting on the gray histogram of the video frame to obtain the gray distribution intervals of the soil area and the environmental background; according to the relationship between the gray value and the gray distribution intervals of the soil area and the environmental background, dividing the pixel points into three categories, and the three obtained categories respectively represent the pixel points in the environmental background, the soil area, and the seedling area; determining the growth attention indicators of the pixel points according to the category to which the pixel points belong and the distance between their gray values and the boundaries of each distribution interval; determining the dynamic spatio-temporal window of the pixel points according to the category to which the pixel points belong and the gradient characteristics of the pixel points, and the dynamic spatio-temporal window is composed of local windows corresponding to the pixel points in multiple video frames on the time axis.

[0008] Preferably, the obtaining of the gray distribution intervals of the soil area and the environmental background includes: for all gray values in the gray histogram of the video frame whose frequencies are not equal to 0, recording the minimum value and the maximum value among them as the minimum gray value and the maximum gray value ; performing Gaussian fitting on the gray values within the range of , and recording the mean value and the standard deviation in the obtained Gaussian fitting result as and , and taking as the gray distribution interval of the soil area; performing Gaussian fitting on the gray values within the range of to obtain the gray distribution interval of the environmental background, and recording the mean value and the standard deviation in the obtained Gaussian fitting result as and , and taking as the gray distribution interval of the environmental background; wherein, and are preset gray boundaries.

[0009] Preferably, the dividing of the pixel points into three categories includes: dividing the pixel points whose gray values are within the gray distribution interval of the environmental background into the first category; dividing the pixel points whose gray values and local background gray values are both within the gray distribution interval of the soil area into the second category; dividing the pixel points whose gray values are within the gray distribution interval of the soil area but the local background gray values are not within the gray distribution interval of the soil area into the third category; dividing the pixel points whose gray values are within the range of into the third category, is the right boundary of the gray-scale distribution interval of the soil area, is the left boundary of the gray-scale distribution interval of the environmental background.

[0010] Preferably, the process of obtaining the local background gray-scale value of the pixel point includes: taking the pixel point as the center, obtaining a local area with a size equal to where is a positive integer greater than 2 and is an odd number; continuously expanding the size of the local area until at least 8 pixel points with a gray-scale difference greater than 10 from the pixel point can be found in the obtained local area, then stopping the expansion operation, and taking the local area obtained at this time as the local background area of the pixel point; where the gray-scale difference refers to the absolute value of the difference between the gray-scale values of two pixel points.

[0011] Preferably, the determination of the growth attention index of the pixel point includes: for the pixel points in the first category, the growth attention index of the pixel point is equal to 0.25; for the pixel points in the second category, the calculation formula of the growth attention index of the pixel point is: ; for the pixel points in the third category, the calculation formula of the growth attention index of the pixel point is: ; in the formula, represents the growth attention index of the pixel point, represents taking the minimum value, is the exponential function with the natural constant as the base, represents the local background gray-scale value of the pixel point, represents the gray-scale value of the pixel point, is the gray-scale distribution interval of the soil area, is the left boundary of the gray-scale distribution interval of the environmental background.

[0012] Preferably, the process of obtaining the local window corresponding to the pixel point in multiple video frames on the time axis includes: for the pixel points in the first or second category, recording the coordinates of the pixel point as , then the local window corresponding to the pixel point in each video frame on the time axis is a rectangular area centered on the pixel point with coordinates in each video frame and with a size equal to 11×11 or 9×9; for the pixel points in the third category, taking it as the target pixel point, recording the video frame where the target pixel point is located as video frame , obtaining the local window corresponding to the target pixel point in video frame , , including: taking the product of the theoretical growth rate of the seedling and the time interval between video frame and video frame as video frame The corresponding growth displacement amount; combining the relationship between the gray value of the target pixel and the gray value of its local background, and determining the displacement vector of the target pixel in the video frame according to the growth displacement amount and the gradient direction of the target pixel in; according to the displacement vector, move the target pixel to obtain the located pixel in the video frame ; then the local window corresponding to the target pixel in the video frame is a rectangular area centered on the located pixel and with a size equal to 7×7, and it is required that two parallel sides of the rectangular area are parallel to the gradient direction of the target pixel, and the other two parallel sides are perpendicular to the gradient direction of the target pixel.

[0013] Preferably, the process of obtaining the displacement vector of the target pixel in the video frame includes: when , if the gray value of the target pixel is less than the gray value of its local background, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to the gray value of its local background, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel; when , if the gray value of the target pixel is less than the gray value of its local background, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to the gray value of its local background, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel; and the modulus of the displacement vector is equal to the growth displacement amount.

[0014] Preferably, the motion filtering of the dynamic growth video according to the dynamic spatio-temporal window and growth attention index of the pixel includes: for the video frames of the dynamic growth video to be processed, smoothing the pixels in the input video frame through the dynamic spatio-temporal window of the pixel, and assuming that each pixel in the output video frame is obtained by displacing the corresponding input video frame in the displacement field. When constructing the energy function, weighting each term in the energy function through the growth attention index of the pixel, and obtaining the displacement field by minimizing the energy function; using the obtained displacement field to displace the input video frame to obtain the corresponding output video frame, and combining all the output video frames to obtain the filtered dynamic growth video.

[0015] Preferably, the dynamic traits of seedling growth refer to the growth significance at each stage, and the specific acquisition includes: dividing all video frames into multiple stages, and each stage corresponds to video frames, is the shooting time interval between every two adjacent video frames in the dynamic growth video, = 6, represents rounding up; among all the video frames corresponding to each period, the height of the seedlings in the first video frame and the last video are respectively used as the height at the start and end of each period, and the height difference between the end and start of each period is used as the growth amount of the seedlings in each period; obtain the growth amount of the seedlings in each period, and take the ratio of the average growth amount of all the seedlings in the same period to the maximum value of the growth amounts of all the seedlings in all periods as the growth significance of the corresponding period.

[0016] In a second aspect, the present invention provides a seedling growth dynamic trait extraction system, including a processor and a memory, and the memory stores computer program instructions, which, when executed by the processor, implement the above-mentioned seedling growth dynamic trait extraction method.

[0017] By adopting the above technical solution, the above-mentioned seedling growth dynamic trait extraction method is generated into a computer program and stored in the memory to be loaded and executed by the processor, so as to manufacture a terminal device according to the memory and the processor, which is convenient to use.

[0018] The beneficial effects of the present invention are as follows:

[0019] According to the characteristics of pixel points, the present invention sets different dynamic spatio-temporal windows for pixel points with different characteristics, which can more accurately identify motion noise irrelevant to seedling growth and reduce interference with video analysis. At the same time, the dynamic adjustment of the window can provide higher flexibility and accuracy, thereby improving the accuracy of the seedling growth dynamic traits obtained through the analysis of the dynamic growth video of the seedling raising process; at the same time, the present invention calculates and sets the growth attention index of pixel points according to the class to which the pixel points belong, and sets different growth attention indexes for pixel points representing the environmental background, soil area and seedling area. When performing motion filtering on the dynamic growth video, each item in the energy function is weighted by the growth attention index of the pixel points, so that the energy field obtained by solving the energy function focuses on the key pixel points of seedling growth, that is, the pixel points in the seedling area, improving the clarity of the pixel points in the seedling area in the filtered dynamic growth video, and further improving the accuracy and reliability of the extraction of growth dynamic traits. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0021] Figure 1 is a flowchart schematically showing a method for extracting dynamic traits of seedling growth in the present invention;

[0022] Figure 2 is a schematic diagram schematically showing a spatio-temporal window;

[0023] Figure 3 is a flowchart schematically showing setting a dynamic spatio-temporal window and growth attention indicators for each pixel point in a video frame in step S2;

[0024] Figure 4 is a schematic diagram of a video frame of a dynamic growth video of the seedling raising process;

[0025] Figure 5 is schematically showing Figure 4 a schematic diagram of the grayscale histogram of the video frame shown;

[0026] Figure 6 is a schematic diagram schematically showing a dynamic spatio-temporal window. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0028] Next, the detailed implementation manners of the present invention will be described in detail in conjunction with the accompanying drawings.

[0029] An embodiment of the present invention discloses a method for extracting dynamic traits of seedling growth. Referring to Figure 1 , it includes steps S1 to S3:

[0030] S1. Obtain a dynamic growth video of the seedling raising process through time-lapse photography.

[0031] Seedling raising is to cultivate seedlings, which refers to cultivating seedlings in a nursery, hotbed or greenhouse for transplanting to the land for planting; performing time-lapse photography on the growth process of the seedlings to obtain a dynamic growth video of the seedling raising process, which can be used as visual materials for extracting the growth traits of the seedlings.

[0032] The specific steps for obtaining a dynamic growth video of the seedling raising process through time-lapse photography are as follows:

[0033] 1. Select a camera with time-lapse photography function, including but not limited to a single-lens reflex or mirrorless camera that can manually set the shutter speed and interval shooting; fix the camera with a tripod to avoid shaking during the shooting process.

[0034] 2. During the seedling raising process, to avoid the influence brought by the change of natural light and ensure uniform and stable illumination in the seedling growth environment, artificial lighting (such as LED grow lights) is used; monitor and maintain appropriate temperature and humidity to provide a good growth environment for the seedlings, so as to obtain a more natural growth process.

[0035] 3. Set the shooting time interval. In this embodiment, the shooting time interval is equal to 30 minutes; then the shooting time interval between every two adjacent video frames in the subsequent obtained dynamic growth video is equal to 30 minutes.

[0036] 4. Set the frame rate. Arrange all the images obtained by time-lapse photography into a dynamic growth video of the seedling raising process according to the set frame rate, so as to form a smooth dynamic effect; in this embodiment, the frame rate is equal to 30 fps (frames per second), and fps is the unit of the frame rate, indicating the number of frames displayed per second.

[0037] S2. For the video frames in the dynamic growth video, set the dynamic spatio-temporal window and growth attention index of the pixel points; according to the dynamic spatio-temporal window and growth attention index of the pixel points, perform motion filtering on the dynamic growth video to obtain the filtered dynamic growth video.

[0038] It should be noted that in the dynamic growth video of the seedling raising process taken by time-lapse photography, long-term changes (such as seedling growth) and short-term changes (such as light changes, camera jitter, etc.) are mixed together, and the short-term changes are manifested as the tremors and vibrations of the video, which will interfere with the analysis and understanding of the seedling growth state; therefore, use motion filtering technology to remove the interference of short-term changes in the video, such as light changes, slight camera jitter, etc., and highlight the long-term growth changes of the seedlings.

[0039] The specific implementation steps of motion filtering are as follows: for the video frames of the dynamic growth video to be processed, construct a spatio-temporal window in time and space, smooth the pixel points in the input video frame through the spatio-temporal window, and assume that each pixel point in the output video frame is obtained by displacing its corresponding input video frame in the displacement field, and obtain the displacement field by minimizing the energy function; use the obtained displacement field to displace the input video frame to obtain the corresponding output video frame, and combine all the output video frames to obtain the filtered dynamic growth video; in this way, decompose the actions in the video frames of the dynamic growth video into short-term changes and long-term changes, filter out the short-term changes, and enhance the long-term changes to achieve the motion filtering process of the dynamic growth video.

[0040] Among them, the energy function has three terms: the first term characterizes the fidelity, indicating that the output video frame is displaced from the input video frame, and requires the output video frame to be highly similar to the input video frame; the second term characterizes the temporal similarity, indicating that the output video frame has local smoothness within the spatio-temporal window; the third term is a regularization term, indicating that the displacement field of pixel points has local smoothness.

[0041] Among them, the spatio-temporal window is composed of local windows of the same size corresponding to a plurality of video frames of pixel points on the time axis. A schematic diagram of the spatio-temporal window is shown in Figure 2 shown.

[0042] The motion filtering adopted is the method of "Motion Denoising for Time-Lapse Photography" proposed by Michael Rubinstein et al. in the journal "IEEE Computer Vision and Pattern Recognition (CVPR), June 2011" in 2011; therefore, the motion filtering adopted in the present invention is a well-known technology and will not be elaborated here.

[0043] In motion filtering, the size and position of the spatio-temporal window determine the time dimension and space dimension considered in the filtering process, and thus affect the noise recognition and removal effects: if the spatio-temporal window is set too small, it may lead to over-filtering, that is, misjudging some normal dynamic changes as noise, thus losing important growth information. If the spatio-temporal window is set too large, it may filter out smaller motion details, resulting in insufficient filtering effect, still retaining some unnecessary noise, and at the same time may make the filtered picture blurred; therefore, the setting of the spatio-temporal window has a significant impact on the motion filtering effect; in the existing motion filtering, the same spatio-temporal window is set for all pixel points, which affects the motion filtering effect and thus affects the accuracy of the analysis and understanding of the growth state of seedlings.

[0044] Therefore, properly and dynamically setting the spatio-temporal window of pixel points is the key to achieving effective motion filtering, which can help researchers better understand and analyze the growth process of seedling plants.

[0045] In summary, according to the characteristics of pixel points, the present invention sets different dynamic spatio-temporal windows for pixel points with different characteristics, calculates the growth attention index of pixel points, replaces the fixed spatio-temporal window with a dynamic spatio-temporal window when performing motion filtering on the dynamic growth video, and weights each term in the energy function through the growth attention index of pixel points when constructing the energy function; by dynamically adjusting the size of the spatio-temporal window according to the characteristics of pixel points, it is possible to more accurately identify the motion noise irrelevant to the growth of seedlings and reduce the interference to video analysis; at the same time, dynamically adjusting the window can provide higher flexibility and accuracy, thereby improving the accuracy of growth analysis.

[0046] The flowchart of setting the dynamic spatiotemporal window and growth focus index for each pixel in the video frame in step S2 is shown in FIG. Figure 3 , including steps S201 to S204, specifically:

[0047] S201, performing Gaussian fitting on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background.

[0048] It should be noted that in the dynamic video of seedling growth, it is crucial to accurately distinguish between the soil, environmental background and the seedlings themselves. By distinguishing the pixels in the seedling area from the pixels in the soil area and the environmental background, more attention can be paid to the dynamic growth information of the seedlings, such as the stretching of leaves and the increase in plant height.

[0049] The video frames of the dynamic growth video include seedlings, soil and environment. In the scene of seedling growth, the grayscale values ​​of pixels in the soil area and the environmental background are usually relatively concentrated and narrowly distributed due to factors such as their material and color. For example, the soil often appears darker gray or brown, and the corresponding grayscale values ​​are concentrated in a specific interval of the grayscale histogram close to the origin, while the environmental background has a brighter grayscale value, thereby forming a relatively independent distribution area of ​​the original origin on the grayscale histogram; moreover, the soil area is darker and the environmental background is brighter, that is, the grayscale value of the pixel in the soil area is smaller, while the grayscale value of the pixel in the environmental background is larger, and the pixel in the seedling area is between the two. Therefore, the present invention performs Gaussian fitting on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background.

[0050] Get the grayscale histogram of the video frame. The grayscale histogram can intuitively show the number distribution of pixels corresponding to each grayscale value in the video frame. The horizontal axis of the grayscale histogram is the grayscale value, and the vertical axis is the frequency of each grayscale value. The frequency refers to the gray value in the video frame equal to The number of times a pixel appears.

[0051] For example, a schematic diagram of a video frame of a dynamic growth video of a seedling raising process is as follows: Figure 4 As shown, Figure 4 The schematic diagram of the grayscale histogram of the video frame shown is as follows Figure 5 shown.

[0052] Gaussian fitting is performed on the grayscale histogram of the video frame to obtain the grayscale distribution interval of the soil area and the grayscale distribution interval of the environmental background. The specific method is as follows: for all grayscale values ​​whose frequency is not equal to 0 in the grayscale histogram, the minimum and maximum values ​​are recorded as the minimum grayscale value, respectively. and the maximum gray value ; For the gray values within , perform Gaussian fitting, and denote the mean and standard deviation in the obtained Gaussian fitting results as and respectively. Take as the gray distribution interval of the soil area; for the gray values within , perform Gaussian fitting to obtain the gray distribution interval of the environmental background, and denote the mean and standard deviation in the obtained Gaussian fitting results as and respectively. Take as the gray distribution interval of the environmental background.

[0053] Among them, and are preset gray boundaries. Since the preset gray boundary is used to limit the gray distribution interval of the soil area, and the preset gray boundary is used to limit the gray distribution interval of the environmental background area. The soil area is darker and the environmental background is lighter. That is to say, the gray values of the pixel points in the soil area are smaller, while the gray values of the pixel points in the environmental background are larger. Therefore, ; The specific values of the preset gray boundaries and can be set according to the actual application scenario and requirements, and the value range of the preset gray boundary is , and the value range of the preset gray boundary is . Therefore, in this embodiment, the preset gray boundary is set to 40, and the preset gray boundary is set to 120.

[0054] S202. Divide the pixel points into three categories according to the relationship between the gray value and the gray distribution intervals of the soil area and the environmental background.

[0055] It should be noted that in the video frame, the color of the soil area is darker, the color of the seedling area is medium, and the color of the environmental background is lighter. Therefore, the gray values of the pixel points belonging to the soil area, the pixel points belonging to the seedling area, and the pixel points belonging to the environmental background increase in sequence. According to this feature, the pixel points belonging to the soil area, the pixel points belonging to the seedling area, and the pixel points belonging to the environmental background can be distinguished.

[0056] Specifically, there are some pixel points in the seedling area with darker colors, resulting in smaller gray values for these pixel points, which fall within the gray distribution range of the soil area. To distinguish these pixel points, the feature of the local background gray value of the pixel point is introduced. If the local background gray value of the pixel point also falls within the gray distribution range of the soil area, it can be determined that the pixel point belongs to the soil area.

[0057] According to the relationship between the gray value and the gray distribution ranges of the soil area and the environmental background, the pixel points are divided into three categories. Among them, the first category represents the pixel points in the environmental background, the second category represents the pixel points in the soil area, and the third category represents the pixel points in the seedling area. The specific method is as follows:

[0058] 1. If the gray value of the pixel point is within the gray distribution range of the environmental background it indicates that the pixel point is a pixel point in the environmental background. Therefore, the pixel points with gray values within the gray distribution range of the environmental background are classified into the first category.

[0059] 2. If the gray value of the pixel point is within the gray distribution range of the soil area and the local background gray value of the pixel point is also within the gray distribution range of the soil area, it indicates that the pixel point is a pixel point in the soil area. Therefore, the pixel points with both gray values and local background gray values within the gray distribution range of the soil area are classified into the second category.

[0060] 3. If the gray value of the pixel point is within the gray distribution range of the soil area while the local background gray value of the pixel point is not within the gray distribution range of the soil area, it indicates that the pixel point is a pixel point in the seedling area. Therefore, the pixel points with gray values within the gray distribution range of the soil area while the local background gray values are not within the gray distribution range of the soil area are classified into the third category.

[0061] 4. is the gray range between the gray distribution range of the soil area and the gray distribution range of the environmental background, and it is also the gray distribution range of most pixel points in the seedling area. If the gray value of the pixel point is within it indicates that the pixel point is a pixel point in the seedling area. Therefore, the pixel points with gray values within are classified into the third category.

[0062] Among them, the method for obtaining the local background gray value of the pixel point is as follows: Taking the pixel point as the center, a local area with a size equal to is obtained, where is a positive integer greater than 2 andis singular; continuously expand the size of the local area until at least 8 pixels with a gray-scale difference greater than 10 from the pixel point are found in the obtained local area, then stop the expansion operation, and take the local area obtained at this time as the local background area of the pixel point; the gray-scale difference refers to the absolute value of the difference between the gray-scale values of two pixel points.

[0063] S203. Determine the growth attention index of the pixel point according to the class to which the pixel point belongs and the gray-scale value of the pixel point.

[0064] It should be noted that the pixel points close to the seedling often contain key information for seedling growth, such as the edge of the leaf, the junction of the seedling stem and the soil, etc.; taking the leaf edge as an example, as the seedling grows, the leaf gradually unfolds and expands, and the pixel points near its edge will show changes in gray-scale values or other characteristic values. When the leaf gradually unfolds from a curled state to a flat state, the pixel points near the edge change first, from the gray-scale value similar to that inside the leaf in the curled state to a gray-scale value or color value with an obvious difference from the soil background. These changes can sensitively reflect the growth dynamics of the seedling leaf. Therefore, by setting the growth attention index of these pixel points larger, these key information can be captured more preferentially, so as to more accurately monitor and analyze the growth of the seedling.

[0065] 1. For the pixel points in the environmental background, that is, the pixel points in the first class, the growth attention index of the pixel point is a fixed value, and the growth attention index of the pixel point is equal to 0.25.

[0066] 2. For the pixel points in the soil area, that is, the pixel points in the second class, the calculation formula of the growth attention index of the pixel point is as follows:

[0067] ;

[0068] In the formula, represents the growth attention index of the pixel point, represents taking the minimum value, represents the local background gray-scale value of the pixel point, is the gray-scale distribution interval of the soil area, is the right boundary of the gray-scale distribution interval of the soil area.

[0069] Among them, is the gray-scale distribution interval of the soil area, is the gray-scale distribution interval of most pixel points in the seedling area, is the key gray-scale value for distinguishing the soil area and the seedling area. Therefore, for the pixel points in the soil area, that is, the pixel points in the second class, the local background gray-scale value of the pixel point and the key gray-scale value The farther the distance, the The larger it is, the more likely that the pixel is a pixel near the seedling area. Therefore, compared with other pixels belonging to the soil area, its growth concern index should be set larger.

[0070] 3. For the pixels in the seedling area, that is, the pixels in the third category, the calculation formula for the growth concern index of the pixels is as follows:

[0071] ;

[0072] In the formula, Represents the growth attention index of the pixel point, Represents the gray value of a pixel. represents an exponential function with a natural constant as base, is the grayscale distribution interval of the soil area The right boundary of is the grayscale distribution interval of the environmental background The left border of .

[0073] Among them, the gray distribution range of the soil area The right border And the grayscale distribution range of the environmental background Left border of , which constitutes the grayscale distribution range of most pixels in the seedling area ,and is the key gray value to distinguish the soil area from the seedling area. is the key grayscale value to distinguish the environmental background from the seedling area; the grayscale value of the stem part belonging to the seedling area is larger, while the grayscale value of the leaf part belonging to the seedling area is smaller; in the dynamic growth video of the seedling, more attention is paid to the growth of the stem than the growth of the leaf, because the growth of the stem is more stable than that of the leaf, and can better reflect the overall growth trend of the seedling; therefore, for the pixel point in the seedling area, that is, the pixel point in the third category, when the grayscale value of the pixel point is equal to the key grayscale value to distinguish the environmental background from the seedling area Distance Smaller, and the key gray value to distinguish the soil area from the seedling area Distance When it is larger, it means that the pixel point is more likely to belong to the stem part of the seedling area and less likely to belong to the leaf part of the seedling area, so the growth attention index of the pixel point is set to a larger value.

[0074] It should be noted that according to the class to which the pixel belongs, the present invention calculates and sets the growth attention index of the pixel, and sets different growth attention indexes for the pixels representing the environmental background, the soil area, and the seedling area. When performing motion filtering on the dynamic growth video, each item in the energy function is weighted by the growth attention index of the pixel, so that the energy field obtained by solving the energy function specifically focuses on the key pixels of the seedling growth, that is, the pixels in the seedling area, improving the clarity of the pixels in the seedling area in the filtered dynamic growth video, and further improving the accuracy and reliability of the extraction of the growth dynamic traits.

[0075] S204. Determine the dynamic spatio-temporal window of the pixel according to the class to which the pixel belongs and the gradient feature of the pixel.

[0076] Among them, the dynamic spatio-temporal window of the pixel is composed of local windows corresponding to the pixel in multiple video frames on the time axis, and the multiple video frames on the time axis include video frame 、video frame 、video frame 、video frame 、video frame , where video frame is the video frame where the pixel is located, then video frames and video frame are the two adjacent video frames before video frame , and video frames and video frame are the two adjacent video frames after video frame ; the schematic diagram of the dynamic spatio-temporal window of the pixel is as shown in Figure 6 .

[0077] Therefore, it is necessary to determine the local window corresponding to the pixel in multiple video frames on the time axis in the following. The specific method is as follows:

[0078] 1. For the pixels in the environmental background, that is, the pixels in the first class, record the coordinates of the pixel as , then the local window corresponding to the pixel in each video frame on the time axis is a rectangular area centered on the pixel with coordinates in each video frame and with a size equal to 11×11.

[0079] 2. For the pixels in the soil area, that is, the pixels in the second class, record the coordinates of the pixel as , then the local window corresponding to the pixel in each video frame on the time axis is a rectangular area centered on the pixel with coordinates in each video frame and with a size equal to 9×9.

[0080] It should be noted that for the pixel points in the seedling area, due to the growth of the seedlings, the positions of the corresponding pixel points in adjacent video frames change. However, for the pixel points in the environmental background and soil area, they are basically not affected by the growth of the seedlings, and the positions of the corresponding pixel points in adjacent video frames do not change. Therefore, the positions of the local windows corresponding to the pixel points in the environmental background and soil area in each video frame on the time axis are fixed, while the positions of the local windows corresponding to the pixel points in the seedling area in different video frames on the time axis are different. It is necessary to calculate the growth displacement according to the theoretical growth rate of the seedlings and the time interval between video frames and video frame to set the positions of the local windows corresponding to the pixel points in the seedling area in different video frames on the time axis.

[0081] 3. For the pixel points in the seedling area, that is, the pixel points in the third category, regard them as target pixel points, and record the video frame where the target pixel points are located as video frame , and the method for obtaining the local window corresponding to the target pixel points in video frame is as follows, where :

[0082] 3.1. Calculate the gradient features of the target pixel points through the Sobel operator. The gradient features include the gradient direction and the gradient magnitude.

[0083] 3.2. Calculate the growth displacement corresponding to video frame according to the theoretical growth rate of the seedlings and the time interval between video frame and video frame .

[0084] Among them, the time interval between video frame and video frame is equal to , represents the shooting time interval between every two adjacent video frames in the dynamic growth video; the theoretical growth rate refers to the length of the seedling growth per unit time, and for the length of the seedling growth, its measurement unit is the target pixel point. Exemplarily, the length of the seedling growth per unit time is 3 target pixel points. Therefore, the theoretical growth rate is 3; then the growth displacement corresponding to video frame is equal to the product of the time interval between video frames and the theoretical growth rate.

[0085] 3.3. Combine the relationship between the gray value of the target pixel point and the gray value of its local background, and determine the displacement vector of the target pixel point in video frame according to the growth displacement and the gradient direction of the target pixel point.

[0086] When When it is, that is, the video frame is the video frame When it is the video frame after that, determine the magnitude relationship between the gray value of the target pixel and its local background gray value: If the gray value of the target pixel is less than its local background gray value, the growth direction of the seedling area corresponding to the target pixel is opposite to the gradient direction of the target pixel. At this time, search for the target pixel corresponding to this target pixel in the video frame along the opposite direction of the gradient direction. Therefore, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel, and the magnitude of the displacement vector is equal to the growth displacement; If the gray value of the target pixel is greater than or equal to its local background gray value, the growth direction of the seedling area corresponding to the target pixel is the same as the gradient direction of the target pixel. At this time, search for the target pixel corresponding to this target pixel in the video frame along the gradient direction. Therefore, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel, and the magnitude of the displacement vector is equal to the growth displacement.

[0087] When it is, that is, the video frame is the video frame When it is the video frame before that, determine the magnitude relationship between the gray value of the target pixel and its local background gray value: If the gray value of the target pixel is less than its local background gray value, the growth direction of the seedling area corresponding to the target pixel is the same as the gradient direction of the target pixel. At this time, search for the target pixel corresponding to this target pixel in the video frame along the gradient direction. Therefore, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel, and the magnitude of the displacement vector is equal to the growth displacement; If the gray value of the target pixel is greater than or equal to its local background gray value, the growth direction of the seedling area corresponding to the target pixel is opposite to the gradient direction of the target pixel. At this time, search for the target pixel corresponding to this target pixel in the video frame along the opposite direction of the gradient direction. Therefore, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel, and the magnitude of the displacement vector is equal to the growth displacement.

[0088] 3.4. Move the target pixel according to the displacement vector of the target pixel in the video frame to obtain the located pixel in the video frame .

[0089] When When it is, that is, for the video frame where the target pixel is located , the displacement vector does not exist. Therefore, the located pixel in the video frame is the target pixel.

[0090] When it is, that is, for the video frame adjacent to the video frame where the target pixel is located , the target pixel is moved according to the displacement vector in the video frame . The moving direction is the direction of the displacement vector, and the moving distance is equal to the modulus of the displacement vector, obtaining the located pixel in the video frame .

[0091] 3.5. According to the located pixel in the video frame , obtain the local window corresponding to the target pixel in the video frame . Then, the local window corresponding to the target pixel in the video frame is a rectangular area centered on the located pixel in each video frame and with a size equal to 7×7. It is required that two parallel sides in the rectangular area are parallel to the gradient direction of the target pixel, and the other two parallel sides are perpendicular to the gradient direction of the target pixel.

[0092] It should be noted that according to the characteristics of the pixel, different dynamic spatio-temporal windows are set for pixels with different characteristics, which can more accurately identify the motion noise unrelated to seedling growth, reduce the interference to video analysis. At the same time, the dynamic adjustment of the window can provide higher flexibility and accuracy, thereby improving the accuracy of the seedling growth dynamic traits obtained through the dynamic growth video analysis of the seedling raising process.

[0093] S3. Analyze the filtered dynamic growth video and extract the seedling growth dynamic traits.

[0094] The seedling growth dynamic traits refer to the growth significance at each stage. The specific acquisition includes:[[]]

[0095] Analyze the filtered dynamic growth video: Divide all video frames into multiple stages, and each stage corresponds to video frames; In all video frames corresponding to each stage, take the height of the seedling in the first video frame and the last video frame respectively as the height at the beginning and end of each stage, and take the height difference between the end and the beginning of each stage as the growth amount of the seedling in each stage; Obtain the growth amount of the seedling in each stage, and take the ratio of the average growth amount of all seedlings in the same stage to the maximum value of the growth amounts of all seedlings in all stages as the growth significance of the corresponding stage.

[0096] is the shooting time interval between every two adjacent video frames in the dynamic growth video, represents rounding up; in addition, the implementer can set the value of according to the actual implementation situation, for example = 6.

[0097] An embodiment of the present invention also discloses a seedling growth dynamic trait extraction system, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a seedling growth dynamic trait extraction method according to the present invention is implemented.

[0098] The above system also includes other components well known to those skilled in the art such as a communication bus and a communication interface, and their settings and functions are known in the art, so they will not be elaborated here.

Claims

1. A method for extracting dynamic traits of seedling growth, characterized in that, Including: Obtaining a dynamic growth video of the seedling raising process through time-lapse photography; for the video frames in the dynamic growth video, performing motion filtering on the dynamic growth video according to the dynamic spatio-temporal window of pixel points and the growth attention index to obtain the filtered dynamic growth video; analyzing the filtered dynamic growth video to extract the dynamic traits of seedling growth. Among them, the method for obtaining the dynamic spatio-temporal window of pixel points and the growth attention index is as follows: Perform Gaussian fitting on the grayscale histogram of the video frame to obtain the grayscale distribution intervals of the soil area and the environmental background; according to the relationship between the grayscale value and the grayscale distribution intervals of the soil area and the environmental background, divide the pixel points into three categories, including: dividing the pixel points with grayscale values within the grayscale distribution interval of the environmental background into the first category; dividing the pixel points with both grayscale values and local background grayscale values within the grayscale distribution interval of the soil area into the second category; dividing the pixel points with grayscale values within the grayscale distribution interval of the soil area while the local background grayscale values are not within the grayscale distribution interval of the soil area into the third category, and dividing the pixel points with grayscale values within into the third category, is the right boundary of the grayscale distribution interval of the soil area, is the left boundary of the grayscale distribution interval of the environmental background; the three obtained categories respectively represent the pixel points in the environmental background, the soil area, and the seedling area; determine the growth attention index of the pixel points according to the category to which the pixel points belong, their grayscale values, and the distances from the boundaries of the respective distribution intervals; determine the dynamic spatio-temporal window of the pixel points according to the category to which the pixel points belong and the gradient characteristics of the pixel points, and the dynamic spatio-temporal window is composed of local windows corresponding to the pixel points in multiple video frames on the time axis; The process of obtaining the local windows corresponding to a pixel point in multiple video frames on the time axis includes: for the pixel points in the first type or the second type, recording the coordinates of the pixel point as , then the local window corresponding to the pixel point in each video frame on the time axis is a rectangular area centered on the pixel point with coordinates in each video frame and with a size equal to 11×11 or 9×9; for the pixel points in the third type, taking them as target pixel points, recording the video frame where the target pixel point is located as video frame , obtaining the local window corresponding to the target pixel point in video frame , , including: taking the product of the theoretical growth rate of the seedling and the time interval between video frame and video frame as the growth displacement amount corresponding to video frame ; combining the magnitude relationship between the gray value of the target pixel point and its local background gray value, determining the displacement vector of the target pixel point in video frame according to the growth displacement amount and the gradient direction of the target pixel point; moving the target pixel point according to the displacement vector to obtain the located pixel point in video frame ; then the local window corresponding to the target pixel point in video frame is a rectangular area centered on the located pixel point and with a size equal to 7×7, and it is required that two parallel sides in the rectangular area are parallel to the gradient direction of the target pixel point, and the other two parallel sides are perpendicular to the gradient direction of the target pixel point.

2. The method for extracting dynamic traits of seedling growth according to claim 1, wherein The obtaining of the gray-scale distribution interval of the soil area and the environmental background includes: For all gray values in the gray histogram of the video frame whose frequencies are not equal to 0, record the minimum value and the maximum value among them as the minimum gray value and the maximum gray value ; For the gray values within the range, perform Gaussian fitting, and denote the mean and standard deviation in the obtained Gaussian fitting results as and respectively. Take as the gray distribution interval of the soil area; for the gray values within the range, perform Gaussian fitting to obtain the gray distribution interval of the environmental background, and denote the mean and standard deviation in the obtained Gaussian fitting results as and respectively. Take as the gray distribution interval of the environmental background; where and are preset gray boundaries.

3. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that, The obtaining process of the local background gray-scale value of the pixel point includes: Taking the pixel point as the center, obtain a local area with a size equal to . is a positive integer greater than 2 and is an odd number; continuously expand the size of the local area until at least 8 pixel points with a gray - scale difference greater than 10 from the pixel point are found in the obtained local area, then stop the expansion operation, and take the local area obtained at this time as the local background area of the pixel point; Among them, the gray-scale difference refers to the absolute value of the difference between the gray-scale values of two pixel points.

4. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that, The determination of the growth attention index of the pixel point includes: For the pixel points in the first category, the growth attention index of the pixel point is equal to 0.25; For the pixel points in the second category, the calculation formula of the growth attention index of the pixel point is: ； For the pixel points in the third category, the calculation formula of the growth attention index of the pixel point is: ； In the formula, represents the growth concern index of the pixel point, represents taking the minimum value, is the exponential function with the natural constant as the base, represents the local background gray value of the pixel point, represents the gray value of the pixel point, is the gray distribution interval of the soil area, is the left boundary of the gray distribution interval of the environmental background.

5. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that, The process of obtaining the displacement vector of the target pixel point in the video frame includes: When , if the gray value of the target pixel is less than its local background gray value, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel; When , if the gray value of the target pixel is less than its local background gray value, the direction of the displacement vector of the target pixel in the video frame is the same as the gradient direction of the target pixel; if the gray value of the target pixel is greater than or equal to its local background gray value, the direction of the displacement vector of the target pixel in the video frame is opposite to the gradient direction of the target pixel; And the modulus of the displacement vector is equal to the growth displacement amount.

6. The method for extracting dynamic traits of seedling growth according to claim 1, wherein The performing of motion filtering on the dynamic growth video according to the dynamic spatio-temporal window of pixel points and the growth attention index includes: For the video frames of the dynamic growth video to be processed, smoothing the pixel points in the input video frames through the dynamic spatio-temporal window of pixel points, and assuming that each pixel point in the output video frames is obtained by displacing the corresponding input video frames in the displacement field. When constructing the energy function, weighting each term in the energy function through the growth attention index of the pixel point, and obtaining the displacement field by minimizing the energy function; displacing the input video frames by using the obtained displacement field to obtain the corresponding output video frames, and combining all the output video frames to obtain the filtered dynamic growth video.

7. A method for extracting dynamic traits of seedling growth according to claim 1, characterized in that, The dynamic traits of seedling growth refer to the growth significance at each stage, and the specific obtaining includes: Divide all video frames into multiple epochs, each epoch corresponding to a number of video frames, where is the shooting time interval between every two adjacent video frames in the dynamic growth video, = 6, indicating rounding up; In all the video frames corresponding to each stage, taking the height of the seedlings in the first video frame and the last video frame as the height at the beginning and the end of each stage respectively, and taking the height difference between the end and the beginning of each stage as the growth amount of the seedlings in each stage; obtaining the growth amount of the seedlings in each stage, and taking the ratio of the average growth amount of all the seedlings in the same stage to the maximum value of the growth amounts of all the seedlings in all stages as the growth significance of the corresponding stage.

8. A seedling growth dynamic trait extraction system, characterized in that, Including: A processor and a memory, where the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for extracting dynamic traits of seedling growth according to any one of claims 1-7 is implemented.

Images