Corn ear phenotypic analysis method, device and equipment

Abstract

The present application provides a method, device and equipment for phenotypic analysis of corn ears, wherein the method can perform depth processing on the original image of corn ears to obtain a depth map of the corn ears, and through a pyramid fusion strategy, dynamically adjust the segmentation threshold by using local features in depth images of different scales, fuse the corn ear mask images obtained at each layer of the pyramid to obtain a corresponding ear mask image; perform regional diffusion processing based on texture information and depth information on the ear mask image to divide each single ear mask; extract the single ear image by performing single ear mask traversal on the original image of the corn ear; divide the single ear image into a bald tip area and a grain area by using a color information distinction algorithm, thereby obtaining a bald tip area mask and a grain area mask; determine various phenotypic parameters of the corn ear according to the bald tip area mask, the grain area mask and the ear mask, and calculate the ear uniformity and / or the grain compactness.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and in particular to a method, device and equipment for analyzing corn ear phenotype. Background Art

[0002] As more and more studies on plant phenotypes are conducted, more and more solutions for plant phenotype processing are also being developed. For corn, an important food crop, the current processing method for some of its corresponding data is generally to directly determine some three-dimensional image data and directly determine the phenotypic data.

[0003] However, the current method of determining phenotypic data of corn images does not yield accurate phenotypic data, which will affect some analyses of corn phenotypes. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose a method, device and equipment for analyzing corn ear phenotypes to solve or partially solve the above-mentioned technical problems.

[0005] Based on the above purpose, the present application provides a method for analyzing corn ear phenotype, comprising:

[0006] Acquire an original image of a corn ear, and perform depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map;

[0007] A pyramid fusion strategy is adopted for the corn ear depth map to obtain a multi-layer pyramid image, and the multi-layer pyramid images are processed respectively by dynamically adjusting the segmentation threshold using the local features in the multi-scale depth image to obtain mask data corresponding to each layer of the pyramid image, and all the obtained mask images are fused to obtain the corn ear mask data of the corn ear area;

[0008] Performing regional diffusion processing on the ear mask data using texture information and depth information, dividing the ear mask data to obtain divided single ear masks;

[0009] Using the single ear mask data, mask traversal is performed on the original image of the corn ear to extract the single ear image;

[0010] The single ear image is divided into the bald tip area and the grain area by using the color information distinction algorithm, and the bald tip area mask and the grain area mask are obtained;

[0011] Processing is performed according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear;

[0012] The ear uniformity and / or the kernel compactness are determined according to various phenotypic parameters of the corn ear, and the ear uniformity and / or the kernel compactness are output.

[0013] Based on the same inventive concept, the present application also provides a corn ear phenotype analysis device, comprising:

[0014] A depth processing module is configured to obtain an original image of a corn ear, and perform depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map;

[0015] The region division module is configured to adopt a pyramid fusion strategy for the corn ear depth map to obtain a multi-layer pyramid image, and dynamically adjust the segmentation threshold using local features in the multi-scale depth image to process the multi-layer pyramid images respectively to obtain mask data corresponding to each layer of the pyramid image, and fuse all the obtained mask images to obtain the corn ear mask data of the corn ear region;

[0016] The mask division module is configured to perform regional diffusion processing on the ear mask data using texture information and depth information, divide the ear mask data, and obtain a divided single ear mask;

[0017] A single ear segmentation module is configured to use the single ear mask data to perform mask traversal on the original image of the corn ear to extract the single ear image;

[0018] The bald tip and grain division module is configured to divide the single ear image into a bald tip region and a grain region by using a color information distinction algorithm, and obtain a bald tip region mask and a grain region mask;

[0019] A parameter determination module is configured to perform processing according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear;

[0020] The calculation module is configured to determine the uniformity of the corn ear and / or the compactness of the kernels according to various phenotypic parameters of the corn ear, and output the uniformity of the corn ear and / or the compactness of the kernels.

[0021] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0022] From the above, it can be seen that the corn ear phenotypic analysis method, device and equipment provided by the present application can perform deep processing on the original image of the corn ear, and then obtain a depth map of the corn ear, so that a multi-layer pyramid image of different resolutions can be obtained through a pyramid fusion strategy, so that each layer of the pyramid image can be divided into regions according to dynamically adjusted thresholds of different scales, and then the corresponding ear mask data can be obtained by fusion with different weights; then the ear mask data will be subjected to regional diffusion processing based on texture information and depth information, so that the single ear can be better divided and the depth map can be obtained. And divide each single ear mask; then, since the corn ear has two parts, the bald tip and the kernel, in order to better study the corn ear, it is necessary to divide the original image of the corn ear into the bald tip area mask and the kernel area mask through the color information distinction algorithm; finally, according to the bald tip area mask, kernel area mask and cob mask obtained above, the various phenotypic parameters of the corn ear are determined, and then the corresponding cob uniformity and / or kernel compactness are calculated, so that the consistency of the morphological results of the corn ear can be determined based on the cob uniformity, and the kernel filling effect and compactness of the corn ear can be determined based on the kernel compactness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of a method for analyzing corn ear phenotypes according to an embodiment of the present application;

[0025] Figure 2 A schematic diagram of a corn ear depth map according to an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a multi-layer pyramid image according to an embodiment of the present application;

[0027] Figure 4 A schematic diagram of ear mask data of an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a divided ear mask according to an embodiment of the present application;

[0029] Figure 6 This is a schematic image diagram of the bald tip and grain division of an embodiment of the present application;

[0030] Figure 7 A schematic diagram of a grain region mask according to an embodiment of the present application;

[0031] Figure 8 This is a structural block diagram of a corn ear phenotype analysis device according to an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0034] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0035] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0036] The corn ear phenotype analysis method proposed in the present application example is as follows: Figure 1 As shown, the method includes:

[0037] Step 101, obtaining an original image of a corn ear, and performing depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map (eg, Figure 2 as shown).

[0038] In a specific implementation, the original image of the corn ear is an RGB image captured by a camera. The monocular depth estimation model can estimate the distance of each pixel in the image relative to the shooting source using an RGB image under one or only one perspective (for example, the original image of the corn ear).

[0039] Step 102, a pyramid fusion strategy is used on the corn ear depth map to obtain a multi-layer pyramid image (such as Figure 3As shown in ), and use the local features in the multi-scale depth image to dynamically adjust the segmentation threshold to process the multi-layer pyramid images separately, obtain the mask data corresponding to each layer of the pyramid image, and fuse all the obtained mask images to obtain the corn ear mask data of the corn ear area (as shown in Figure 4 as shown).

[0040] In specific implementation, the pyramid fusion strategy can divide the corn ear depth map into multiple levels, so that it can be segmented layer by layer in a manner of reducing resolution (for example, reducing each layer to 1 / 2 of the previous layer), thereby obtaining a multi-layer pyramid image. In this way, the pyramid image of the corresponding scale can be divided into regions according to the dynamically adjusted thresholds corresponding to different scales, and the corn ear region corresponding to each layer of the pyramid image can be divided, thus obtaining the ear mask data.

[0041] Step 103, performing regional diffusion processing on the ear mask data using texture information and depth information, dividing the ear mask data, and obtaining a divided single ear mask (such as Figure 5 as shown).

[0042] In specific implementation, since there may be multiple ears of fruit in the ear mask data obtained in the above steps, it is necessary to divide each ear of fruit according to texture and depth, and then obtain the ear mask of a single ear of fruit after division.

[0043] Step 104, using the single ear mask data, mask traversal is performed on the original image of the corn ear to extract the single ear image.

[0044] Step 105, using a color information differentiation algorithm to divide the single ear image into a bald tip area and a grain area (eg Figure 6 As shown, the upper part is the bald tip area and the lower part is the grain area), and the bald tip area mask and the grain area mask are obtained (as shown in Figure 7 The image shown is a mask of the kernel region).

[0045] In specific implementation, since the bald tip part and the kernel part of the corn are different in color and relatively easy to distinguish, for example, the bald tip part is dark red and the kernel part is yellow; in another example, the bald tip part is white and the kernel part is yellow; in another example, the bald tip part is white and the kernel part is purple-red / purple-black / flowery; in another example, the bald tip part is dark red and the kernel part is purple-red / purple-black / flowery.

[0046] Step 106, performing processing according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear.

[0047] In specific implementation, various phenotypic parameters of corn ears required for corn analysis are calculated based on the bald tip area mask, the grain area mask and / or the ear mask. Specifically, the various phenotypic parameters of these corn ears can be output to a display screen for display, so that users can view the various phenotypic parameters of these corn ears through the display screen.

[0048] Step 107, determining the uniformity of the corn ear and / or the compactness of the kernels according to various phenotypic parameters of the corn ear, and outputting the uniformity of the corn ear and / or the compactness of the kernels.

[0049] In specific implementation, the uniformity of the ears can reflect the consistency of the morphological structure between different ears, and the compactness of the grains can reflect the filling effect of the grains in the ears, thereby determining the maturity and harvest of the corn.

[0050] Through the above scheme, the original image of the corn ear can be deeply processed to obtain the depth map of the corn ear, so that the multi-layer pyramid images of different resolutions can be obtained through the pyramid fusion strategy, so that the pyramid images of each layer can be divided into regions according to the dynamic thresholds obtained at different scales, and then the corresponding ear mask data can be obtained by fusion through different weights; then the ear mask data will be subjected to regional diffusion processing based on texture information and depth information, so that the single ear can be better divided, and then each single ear mask can be divided; then, because the corn ear will have two parts, the bald tip and the kernel, in order to better study the corn ear, it is necessary to divide the original image of the corn ear into the bald tip area mask and the kernel area mask through the color information distinction algorithm; finally, according to the bald tip area mask, the kernel area mask and the ear mask obtained above, the various phenotypic parameters of the corn ear are determined, and then the corresponding ear uniformity and / or kernel compactness are calculated, so that the consistency of the morphological results of the corn ear can be determined based on the ear uniformity, and the kernel filling effect and compactness of the corn ear can be determined based on the kernel compactness.

[0051] In some embodiments, step 101 includes:

[0052] Step 1011, obtaining an original image of a corn ear from the collected image containing the corn ear, smoothing the original image of the corn ear by using a mean filter, and retaining the edge and details of the original image of the corn ear to obtain a denoised original image of the corn ear.

[0053] Step 1012, dividing the denoised original image of corn ears into a plurality of sub-blocks, performing histogram equalization on each small block, and merging the equalized small blocks to obtain a processed original image of corn ears.

[0054] Step 1013, performing depth processing on the processed corn ear original image using a monocular depth estimation model to obtain a corn ear depth map.

[0055] Through the above scheme, denoising can avoid the influence of some graphic noise in the original image of corn ears, making the subsequent processing more accurate, and the denoised original image of corn ears is divided into multiple sub-blocks and then merged after histogram equalization processing, which can weaken the shadow effect caused by the shooting angle.

[0056] In some embodiments, step 102 includes:

[0057] Step 1021, the corn ear depth map is divided into multiple levels by using a pyramid fusion strategy, and the corn ear depth map of each layer is down-sampled by a Gaussian filter to obtain a multi-layer pyramid image.

[0058] Step 1022, segment each layer of the pyramid image using a threshold value of a corresponding scale to obtain a segmentation result of the corresponding scale, perform weighted processing on the segmentation result of each scale, and obtain a final mask image as corn ear mask data of the corn ear region.

[0059] In specific implementation, each scale is assigned a corresponding weight, so that the segmentation result corresponding to each scale can be weighted using the corresponding weight, and a mask is obtained for each scale, so that all masks are integrated together to form the corn ear mask data of the corn ear area.

[0060] Through the above scheme, the corn ear depth map can be better segmented into pyramids, and the masks at the corresponding scales of each layer can be extracted, and more accurate ear mask data can be formed after combination.

[0061] In some embodiments, step 1022 includes:

[0062] Step 10221, dynamically adjust each layer of the pyramid image using an adaptive threshold segmentation algorithm to generate a local adaptive threshold corresponding to the ear region.

[0063] In specific implementation, Represents the image corresponding to the i-th layer of the pyramid, and the corresponding local adaptive threshold of the i-th layer is , the formula is: ,in, Indicates the pixel position at the i-th level of the pyramid The gradient of reflects the depth change rate, Represents the local contrast at the i-th layer of the pyramid, reflecting the local brightness difference of the image. α and β represent weight coefficients, which are used to control the gradient and contrast contribution.

[0064] Step 10222: Segment each layer of the pyramid image using the local adaptive threshold to obtain a binary mask of a corresponding scale of each layer as a segmentation result.

[0065] In specific implementation, the binary mask The formula is: .

[0066] Step 10223, weighting the segmentation results of each layer corresponding to the scale to obtain a mask corresponding to each layer.

[0067] In specific implementation, the mask formula corresponding to each layer is: ,in, is the weight value of the i-th layer, which satisfies , n is the number of pyramid layers, It means aligning the mask of the i-th layer with the high-resolution image of the pyramid by upsampling.

[0068] Step 10224, the masks corresponding to all layers are fused layer by layer from low to high resolution to obtain the corn ear mask data of the corn ear area.

[0069] In specific implementation, , n is the number of pyramid layers, and the layers are gradually fused from low-resolution layers to high-resolution layers according to the above formula until they are fused to the highest-resolution layer of the pyramid, and the ear mask data of the corn ear area is obtained.

[0070] Through the above solution, the ear features corresponding to each layer of pyramid images can be fused together, so that the obtained ear mask data is more accurate.

[0071] In some embodiments, step 103 includes:

[0072] Step 1031, determining a grayscale image containing texture information of the original image of the corn ear, and performing gradient calculation on the grayscale image to obtain a corn ear grayscale gradient matrix .

[0073] In the specific implementation, the Sobel operator is used to calculate the gradient of the grayscale image, thereby obtaining a more accurate ear grayscale gradient matrix .

[0074] The Sobel operator is a discrete difference operator used to calculate the approximate value of the gradient of the image brightness function. Using this operator at any point in the image will generate the corresponding gradient vector or its normal vector.

[0075] Step 1032, calculate the depth gradient of the ear mask to obtain a depth gradient matrix .

[0076] In the specific implementation, the Sobel operator is also used to calculate the depth gradient of the ear mask, thereby obtaining a more accurate depth gradient matrix .

[0077] Step 1033: convert the ear gray gradient matrix and the depth gradient matrix Normalize and fuse using weighted summation to get the gradient fusion result .

[0078] In specific implementation, the calculation formula of gradient fusion result is: , where α and β are the depth gradient matrices and ear gray gradient matrix The weight coefficient is used to adjust the influence of the two on the results.

[0079] Step 1034, selecting the point with the largest gradient change as the seed point according to the gradient fusion result, using the region growing algorithm starting from the seed point, performing region merging and refinement processing based on the similarity criterion, and obtaining a single ear mask.

[0080] Through the above scheme, based on the fusion of the ear grayscale gradient matrix and the depth gradient matrix, and the seed point screening according to the gradient change, it is possible to merge regions according to similarity starting from the seed point, divide each single ear, and obtain the final single ear mask more accurately.

[0081] In some embodiments, step 104 includes:

[0082] The single ear mask image obtained by traversal is subjected to a pixel-by-pixel AND operation with the original image of the corn ear. If the pixel value in the mask is 0, the pixel value at the corresponding position in the original image of the corn ear is also set to 0, that is, black; if the pixel value in the mask is 255, the pixel value at the corresponding position in the original image of the corn ear remains unchanged, and the position coordinates of the contour in the mask image are subjected to an AND operation with the original image to achieve an accurate foreground extraction operation, and obtain an independent image area corresponding to each ear.

[0083] In some embodiments, step 105 includes:

[0084] Step 1051, converting the original image of the corn ear from the RGB color space to the HSV color space, determining the color range of the corn bald tip corresponding to the HSV color space according to the color characteristics of the corn bald tip, using the color range to perform threshold screening, identifying the pixel area matching the corn bald tip, converting the pixel area into a binary mask image, and obtaining a bald tip area mask.

[0085] In the specific implementation, the original image of the corn ear itself is in the RGB (red, green, blue) color space. In order to better capture the corresponding color features, it needs to be converted to the HSV (hue, saturation, value) color space. Then a color threshold is determined, so that the color range of the corn bald tip corresponding to the HSV can be determined according to the color threshold. According to the color range of the corn bald tip, the HSV color space is extracted to obtain the bald tip area mask. The extracted bald tip area is used as the foreground (white), and the other areas are used as the background (black) to form a bald tip area mask.

[0086] Step 1052: extract a green channel image from the RGB color space of the original image of the corn ear, determine an optimal segmentation threshold in the green channel image using a global adaptive threshold segmentation algorithm, and use the optimal segmentation threshold to perform grain segmentation on the original image of the corn ear to obtain a grain region mask.

[0087] In specific implementation, the corresponding global adaptive threshold segmentation algorithm is the Otsu segmentation method, in which the grain area obtained by segmentation is used as the foreground and the other parts are used as the background to obtain the grain area mask.

[0088] Through the above scheme, the bald tip area mask and the grain area mask can be obtained more accurately.

[0089] In some embodiments, step 106 includes:

[0090] Step 1061, determine the minimum bounding box of the ear mask, establish a coordinate system with the major axis of the minimum bounding box as the Y axis and the minor axis of the minimum bounding box as the X axis, use the length value of the minimum bounding box calculated according to the coordinate system as the ear length, and use the calculated width value of the minimum bounding box as the ear width.

[0091] And / or, step 1062, calculating the ear contour area from the number of non-zero pixels in the ear mask. Specifically, determine the area corresponding to each pixel, count the number of non-zero pixels, and multiply the number by the area corresponding to each pixel to obtain the ear contour area.

[0092] And / or, step 1063, in the Y-axis direction of the coordinate system, the average Y value of all contact points of the bald tip area and the grain area is calculated according to the bald tip area mask and the grain area mask, and the average Y value is used as the bald tip dividing line to calculate the bald tip length of the ear.

[0093] And / or, step 1064, using the bald tip area mask and the ear mask to calculate the ear bald tip ratio. Specifically, the ratio of the bald tip area mask to the ear mask is used as the ear bald tip ratio, and the larger the ear bald tip ratio is, the worse the growth effect of the corn is and the worse the maturity is.

[0094] And / or, step 1065, calculating the total number of grains, the average grain length of the grains, the average grain width of the grains and the average grain area of ​​the grains according to the grain area mask.

[0095] In the specific implementation, the watershed algorithm is used to perform mask segmentation on the grain area mask to obtain a single grain mask; then, the smaller masks (i.e., those with an area smaller than a predetermined threshold) among these single grain masks are filtered out, and the number of remaining single grain masks is counted as the total number of grains. The minimum bounding rectangle is determined for each grain mask, and then the area of ​​each grain is calculated based on the long side of the minimum bounding rectangle as the grain length and the short side as the grain width; and the average grain length, average grain width and average grain area of ​​the grain are calculated.

[0096] Step 1065, gridding the minimum bounding box according to the average grain length and average grain width of the grains in the grain region mask, obtaining the rows and columns of the grid, and calculating the center point of each grid in the grid, determining the centroid coordinates of each grain using the grain region mask, calculating the distance between the centroid coordinates of each grain and the center point of the grid, determining the nearest grid to the grain according to the distance, matching the grain to the nearest grid, counting the total number of valid rows in the X-axis direction of the coordinate system in all grids, and calculating the average of the valid grids in each row as the number of grains in the row.

[0097] In the specific implementation, it is ensured that each grain can match a grid, so that it is easy to determine the number of rows of grains. Among them, the length of the ear / the average length of the grain=the number of rows of grains=the number of rows of the grid; the width of the ear / the average width of the grain=the number of columns of grains=the number of columns of the grid.

[0098] Through the above scheme, various phenotypic parameters of corn ears used for corn growth analysis can be accurately determined.

[0099] In some embodiments, step 107 includes:

[0100] Step A1, determining the ear length from various phenotypic parameters of the corn ear (e.g., ), ear width (e.g. ) and ear outline area (e.g. ), and calculated the average ear length, average ear width and average ear outline area.

[0101] The outline area of ​​the ear for , is the bald tip area, is the ear area, and i is the ear number.

[0102] In specific implementation, the corresponding average length of the ear The formula is: n is the number of ears, i is the ear number;

[0103] Average width of bunch The formula is: ;

[0104] Average ear outline area The formula is: .

[0105] Step A2, determining the standard deviation of the ear length, the standard deviation of the ear width and the standard deviation of the ear outline area based on the average value of the ear length, the average value of the ear width and the average value of the ear outline area.

[0106] In specific implementation, the standard deviation of ear length The formula is: ;

[0107] Standard deviation of bunch width The formula is: ;

[0108] Standard deviation of ear outline area The formula is: .

[0109] Step A3, based on the ratio of the standard deviation of ear length to the average value of ear length ( ) as the first discrete coefficient , the formula is: , the ratio of the standard deviation of ear width to the average ear width is taken as the second dispersion coefficient, The formula is: , the ratio of the standard deviation of the ear contour area to the average of the ear contour area is taken as the third dispersion coefficient, The formula is: .

[0110] Step A4, obtaining the uniformity of the ear of fruit by weighted calculation based on the first discrete coefficient, the second discrete coefficient and the third discrete coefficient.

[0111] In specific implementation, the corresponding weight of the first discrete coefficient is , the weight of the second discrete coefficient is , the weight of the third discrete coefficient is ,satisfy The corresponding calculation formula for ear uniformity is ear uniformity = .

[0112] Among them, the uniformity of the bunches is between 0 and 1, and the closer to 1, the more uniform it is.

[0113] and / or,

[0114] Step B1, determining the outline area of ​​the corn ear from various phenotypic parameters of the corn ear. In specific implementation, the outline area of ​​each corn ear i is .

[0115] Step B2, determining the total number of grains, the average grain length of the grains, and the average grain width of the grains from various phenotypic parameters of the corn ear, and calculating the expected total area of ​​the grains.

[0116] In specific implementation, the total number of grains per ear i The length of the kernels in each ear is , the widths are , the average grain length calculated Average grain width , the expected total area of ​​each kernel is .

[0117] Step B3, determining the deviation value between the ear contour area and the expected total area of ​​the grains, and calculating the grain compactness based on the deviation value, the ear contour area and the total number of grains.

[0118] In specific implementation, the corresponding deviation value D is . Kernel compactness = , n is the total number of grains in the corresponding ear.

[0119] Step C, outputting the ear uniformity and / or the grain compactness.

[0120] Through the above scheme, the uniformity of the corn ears and / or the compactness of the grains can be ensured, so that whether the growth of corn ears is uniform can be determined based on the uniformity of the corn ears, and the maturity of the corn ears and grains can be determined based on the compactness of the grains, thereby analyzing the corn yield.

[0121] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0122] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a corn ear phenotype analysis device.

[0124] refer to Figure 8 , the device comprises:

[0125] The depth processing module 201 is configured to obtain an original image of a corn ear, and perform depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map;

[0126] The region division module 202 adopts a pyramid fusion strategy to the corn ear depth map to obtain a multi-layer pyramid image, and dynamically adjusts the segmentation threshold by using the local features in the multi-scale depth image to process the multi-layer pyramid images respectively to obtain mask data corresponding to each layer of the pyramid image, and fuses all the obtained mask images to obtain the corn ear mask data of the corn ear region;

[0127] The mask division module 203 is configured to perform regional diffusion processing on the ear mask data using texture information and depth information, and divide the ear mask data to obtain a divided single ear mask;

[0128] The single ear image segmentation module 204 is configured to use the single ear mask data to perform mask traversal on the original image of the corn ear to extract the single ear image;

[0129] The bald tip and grain division module 205 is configured to divide the single ear image into a bald tip region and a grain region using a color information distinction algorithm to obtain a bald tip region mask and a grain region mask;

[0130] A parameter determination module 206 is configured to process according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear;

[0131] The calculation module 207 is configured to determine the uniformity of the corn ear and / or the compactness of the kernels according to various phenotypic parameters of the corn ear, and output the uniformity of the corn ear and / or the compactness of the kernels.

[0132] In some embodiments, the depth processing module 201 is specifically configured to:

[0133] Obtaining an original image of a corn ear from the collected image containing the corn ear, smoothing the original image of the corn ear by using a mean filter, and retaining the edge and details of the original image of the corn ear to obtain a denoised original image of the corn ear;

[0134] The denoised corn ear original image is divided into a plurality of sub-blocks, and each small block is subjected to histogram equalization, and the equalized small blocks are merged to obtain the processed corn ear original image;

[0135] The processed corn ear original image is depth processed using a monocular depth estimation model to obtain a corn ear depth map.

[0136] In some embodiments, the region division module 202 includes:

[0137] A pyramid division unit is configured to divide the corn ear depth map into multiple levels by using a pyramid fusion strategy, and downsample the corn ear depth map of each level through a Gaussian filter to obtain a multi-layer pyramid image;

[0138] The mask processing unit is configured to segment each layer of the pyramid image using a threshold value of a corresponding scale to obtain a segmentation result of the corresponding scale, and perform weighted processing on the segmentation result of each scale to obtain a final mask image as corn ear mask data of the corn ear area.

[0139] In some embodiments, the mask processing unit is specifically configured to:

[0140] Adaptive threshold segmentation algorithm is used to dynamically adjust each layer of pyramid image to generate a local adaptive threshold corresponding to the ear area.

[0141] Segmenting each layer of the pyramid image using the local adaptive threshold to obtain a binary mask of a corresponding scale of each layer as a segmentation result;

[0142] The segmentation results of the corresponding scale of each layer are weighted to obtain the mask corresponding to each layer;

[0143] The masks corresponding to all layers are fused layer by layer from low to high resolution to obtain the corn ear mask data in the corn ear area.

[0144] In some embodiments, the mask division module 203 is specifically configured to:

[0145] Determine a grayscale image containing texture information of the original image of the corn ear, and perform gradient calculation on the grayscale image to obtain a corn ear grayscale gradient matrix;

[0146] Performing depth gradient calculation on the ear mask to obtain a depth gradient matrix;

[0147] The ear grayscale gradient matrix and the depth gradient matrix are normalized, and fused by weighted summation to obtain a gradient fusion result;

[0148] According to the gradient fusion result, the point with the largest gradient change is selected as the seed point, and the region growing algorithm is used to start from the seed point, and the region merging and refinement processing is performed based on the similarity criterion to obtain a single ear mask.

[0149] In some embodiments, the bald tip and seed dividing module 205 is specifically configured to:

[0150] The original image of the corn ear is converted from the RGB color space to the HSV color space, the color range of the corn bald tip corresponding to the HSV color space is determined according to the color characteristics of the corn bald tip, the color range is used to perform threshold screening, the pixel area matching the corn bald tip is identified, and the pixel area is converted into a binary mask image to obtain a bald tip area mask;

[0151] A green channel image is extracted from the RGB color space of the original image of the corn ear, an optimal segmentation threshold is determined in the green channel image using a global adaptive threshold segmentation algorithm, and the original image of the corn ear is segmented into kernels using the optimal segmentation threshold to obtain a kernel region mask.

[0152] In some embodiments, the parameter determination module 206 is specifically configured to:

[0153] Determine the minimum bounding box of the ear mask, establish a coordinate system with the major axis of the minimum bounding box as the Y axis and the minor axis of the minimum bounding box as the X axis, use the length of the minimum bounding box calculated according to the coordinate system as the ear length, and use the width of the calculated minimum bounding box as the ear width; and / or,

[0154] Calculate the ear contour area by the number of non-zero pixels in the ear mask; and / or,

[0155] In the Y-axis direction of the coordinate system, the average Y value of the contact point between the bald tip area and the grain area is calculated according to the bald tip area mask and the grain area mask, and the average Y value is used as the bald tip dividing line to calculate the bald tip length of the ear; and / or,

[0156] Calculate the percentage of bald tips of ears using the bald tip area mask and the ear mask; and / or,

[0157] According to the grain area mask, the total number of grains, the average grain length, the average grain width and the average grain area of ​​the grains are calculated;

[0158] The minimum bounding box is gridded according to the average grain length and average grain width of the grains in the grain area mask to obtain the rows and columns of the grid, and the center point of each grid in the grid is calculated. The centroid coordinates of each grain are determined by using the grain area mask, and the distance between the centroid coordinates of each grain and the center point of the grid is calculated. The nearest grid to the grain is determined according to the distance, and the grain is matched to the nearest grid. The total number of valid rows in the X-axis direction of the coordinate system in all grids is counted, and the average of the valid grids in each row is calculated as the number of grains in the row.

[0159] In some embodiments, the calculation module 207 is specifically configured to:

[0160] Determine the ear length, ear width and ear outline area from various phenotypic parameters of the corn ear, and calculate the average value of the ear length, the average value of the ear width and the average value of the ear outline area;

[0161] Determine the standard deviation of the ear length, the standard deviation of the ear width and the standard deviation of the ear outline area based on the average value of the ear length, the average value of the ear width and the average value of the ear outline area;

[0162] The ratio of the standard deviation of ear length to the average value of ear length is used as the first dispersion coefficient, the ratio of the standard deviation of ear width to the average value of ear width is used as the second dispersion coefficient, and the ratio of the standard deviation of ear outline area to the average value of ear outline area is used as the third dispersion coefficient;

[0163] Obtaining ear uniformity through weighted calculation according to the first discrete coefficient, the second discrete coefficient and the third discrete coefficient;

[0164] and / or,

[0165] Determining ear outline area from various phenotypic parameters of the corn ear;

[0166] Determine the total number of kernels, the average kernel length, and the average kernel width from various phenotypic parameters of the corn ear, and calculate the expected total kernel area;

[0167] Determining a deviation value between the ear contour area and the expected total area of ​​the grains, and calculating the grain compactness based on the deviation value, the ear contour area and the total number of grains;

[0168] The ear uniformity and / or the kernel compactness are output.

[0169] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0170] The device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0171] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the program.

[0172] Fig. 9 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0173] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0174] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0175] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0176] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0177] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0178] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0179] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0180] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0181] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0182] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0183] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0184] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0185] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0186] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0187] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0188] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0189] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0190] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0191] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for analyzing corn ear phenotype, characterized in that: include: Acquire an original image of a corn ear, and perform depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map; A pyramid fusion strategy is adopted for the corn ear depth map to obtain a multi-layer pyramid image, and the multi-layer pyramid images are processed respectively by dynamically adjusting the segmentation threshold using the local features in the multi-scale depth image to obtain mask data corresponding to each layer of the pyramid image, and all the obtained mask images are fused to obtain the corn ear mask data of the corn ear area; Performing regional diffusion processing on the ear mask data using texture information and depth information, dividing the ear mask data to obtain divided single ear masks; Using the single ear mask data, mask traversal is performed on the original image of the corn ear to extract the single ear image; The single ear image is divided into the bald tip area and the grain area by using the color information distinction algorithm, and the bald tip area mask and the grain area mask are obtained; Processing is performed according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear; The ear uniformity and / or the kernel compactness are determined according to various phenotypic parameters of the corn ear, and the ear uniformity and / or the kernel compactness are output.

2. The method according to claim 1, characterized in that Acquiring an original image of a corn ear, and performing depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map, including: Obtaining an original image of a corn ear from the collected image containing the corn ear, smoothing the original image of the corn ear by using a mean filter, and retaining the edge and details of the original image of the corn ear to obtain a denoised original image of the corn ear; The denoised corn ear original image is divided into a plurality of sub-blocks, and each small block is subjected to histogram equalization, and the equalized small blocks are merged to obtain the processed corn ear original image; The processed corn ear original image is depth processed using a monocular depth estimation model to obtain a corn ear depth map.

3. The method according to claim 1, characterized in that A pyramid fusion strategy is adopted for the corn ear depth map to obtain a multi-layer pyramid image, and the multi-layer pyramid images are processed respectively by dynamically adjusting the segmentation threshold using the local features in the multi-scale depth image to obtain mask data corresponding to each layer of the pyramid image, and all the obtained mask images are fused to obtain the corn ear mask data of the corn ear area, including: The corn ear depth map is divided into multiple levels using a pyramid fusion strategy, and the corn ear depth map of each layer is downsampled by a Gaussian filter to obtain a multi-layer pyramid image; Each layer of the pyramid image is segmented using a threshold value of a corresponding scale to obtain a segmentation result of the corresponding scale. The segmentation result of each scale is weighted to obtain a final mask image as the corn ear mask data of the corn ear area.

4. The method according to claim 3, characterized in that Each layer of the pyramid image is segmented using the threshold of the corresponding scale to obtain the segmentation result of the corresponding scale. The segmentation result of each scale is weighted to obtain the final mask image as the corn ear mask data of the corn ear area, including: Adaptive threshold segmentation algorithm is used to dynamically adjust each layer of pyramid image to generate a local adaptive threshold corresponding to the ear area. Segmenting each layer of the pyramid image using the local adaptive threshold to obtain a binary mask of a corresponding scale of each layer as a segmentation result; The segmentation results of the corresponding scale of each layer are weighted to obtain the mask corresponding to each layer; The masks corresponding to all layers are fused layer by layer from low to high resolution to obtain the corn ear mask data in the corn ear area.

5. The method according to claim 1, characterized in that The ear mask data is subjected to regional diffusion processing using texture information and depth information, and the ear mask data is divided to obtain a divided single ear mask, including: Determine a grayscale image containing texture information of the original image of the corn ear, and perform gradient calculation on the grayscale image to obtain a corn ear grayscale gradient matrix; Performing depth gradient calculation on the ear mask to obtain a depth gradient matrix; The ear grayscale gradient matrix and the depth gradient matrix are normalized, and fused by weighted summation to obtain a gradient fusion result; According to the gradient fusion result, the point with the largest gradient change is selected as the seed point, and the region growing algorithm is used to start from the seed point, and the region merging and refinement processing is performed based on the similarity criterion to obtain a single ear mask.

6. The method according to claim 1, characterized in that The single ear image is divided into the bald tip area and the grain area using the color information distinction algorithm, and the bald tip area mask and the grain area mask are obtained, including: The original image of the corn ear is converted from the RGB color space to the HSV color space, the color range of the corn bald tip corresponding to the HSV color space is determined according to the color characteristics of the corn bald tip, the color range is used to perform threshold screening, the pixel area matching the corn bald tip is identified, and the pixel area is converted into a binary mask image to obtain a bald tip area mask; A green channel image is extracted from the RGB color space of the original image of the corn ear, an optimal segmentation threshold is determined in the green channel image using a global adaptive threshold segmentation algorithm, and the original image of the corn ear is segmented into kernels using the optimal segmentation threshold to obtain a kernel region mask.

7. The method according to claim 1, characterized in that Processing is performed according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear, including: Determine the minimum bounding box of the ear mask, establish a coordinate system with the major axis of the minimum bounding box as the Y axis and the minor axis of the minimum bounding box as the X axis, use the length of the minimum bounding box calculated according to the coordinate system as the ear length, and use the width of the calculated minimum bounding box as the ear width; and / or, Calculate the ear contour area by the number of non-zero pixels in the ear mask; and / or, In the Y-axis direction of the coordinate system, the average Y value of all contact points of the bald tip area and the grain area is calculated according to the bald tip area mask and the grain area mask, and the average Y value is used as the bald tip dividing line to calculate the bald tip length of the ear; and / or, Calculate the percentage of bald tips of ears using the bald tip area mask and the ear mask; and / or, According to the grain area mask, the total number of grains, the average grain length, the average grain width and the average grain area of ​​the grains are calculated; The minimum bounding box is gridded according to the average grain length and average grain width of the grains in the grain area mask to obtain the rows and columns of the grid, and the center point of each grid in the grid is calculated. The centroid coordinates of each grain are determined by using the grain area mask, and the distance between the centroid coordinates of each grain and the center point of the grid is calculated. The nearest grid to the grain is determined according to the distance, and the grain is matched to the nearest grid. The total number of valid rows in the X-axis direction of the coordinate system in all grids is counted, and the average of the valid grids in each row is calculated as the number of grains in the row.

8. The method according to claim 1, characterized in that Determining ear uniformity and / or kernel compactness according to various phenotypic parameters of the corn ear, and outputting the ear uniformity and / or kernel compactness, including: Determine the ear length, ear width and ear outline area from various phenotypic parameters of the corn ear, and calculate the average value of the ear length, the average value of the ear width and the average value of the ear outline area; Determine the standard deviation of the ear length, the standard deviation of the ear width and the standard deviation of the ear outline area based on the average value of the ear length, the average value of the ear width and the average value of the ear outline area; The ratio of the standard deviation of ear length to the average value of ear length is used as the first dispersion coefficient, the ratio of the standard deviation of ear width to the average value of ear width is used as the second dispersion coefficient, and the ratio of the standard deviation of ear outline area to the average value of ear outline area is used as the third dispersion coefficient; Obtaining ear uniformity through weighted calculation according to the first discrete coefficient, the second discrete coefficient and the third discrete coefficient; and / or, Determining ear outline area from various phenotypic parameters of the corn ear; Determine the total number of kernels, the average kernel length, and the average kernel width from various phenotypic parameters of the corn ear, and calculate the expected total kernel area; Determining a deviation value between the ear contour area and the expected total area of ​​the grains, and calculating the grain compactness based on the deviation value, the ear contour area and the total number of grains; The ear uniformity and / or the kernel compactness are output.

9. A corn ear phenotype analysis device, characterized in that: include: A depth processing module is configured to obtain an original image of a corn ear, and perform depth processing on the original image of the corn ear using a monocular depth estimation model to obtain a corn ear depth map; The region division module is configured to adopt a pyramid fusion strategy for the corn ear depth map to obtain a multi-layer pyramid image, and dynamically adjust the segmentation threshold using local features in the multi-scale depth image to process the multi-layer pyramid images respectively to obtain mask data corresponding to each layer of the pyramid image, and fuse all the obtained mask images to obtain the corn ear mask data of the corn ear region; The mask division module is configured to perform regional diffusion processing on the ear mask data using texture information and depth information, divide the ear mask data, and obtain a divided single ear mask; A single ear segmentation module is configured to use the single ear mask data to perform mask traversal on the original image of the corn ear to extract the single ear image; The bald tip and grain division module is configured to divide the single ear image into a bald tip region and a grain region by using a color information distinction algorithm, and obtain a bald tip region mask and a grain region mask; A parameter determination module is configured to perform processing according to the bald tip area mask, the grain area mask and / or the single ear mask to determine various phenotypic parameters of the corn ear; The calculation module is configured to determine the uniformity of the corn ear and / or the compactness of the kernels according to various phenotypic parameters of the corn ear, and output the uniformity of the corn ear and / or the compactness of the kernels.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images