Method and apparatus for extracting spikelets based on three-dimensional CT data

Abstract

The application provides a method and device for extracting ear grains based on three-dimensional CT data. The method comprises the following steps: obtaining three-dimensional CT data of a plant, slicing the three-dimensional CT data of the plant to obtain a plurality of slice images; identifying and analyzing each slice image by using a pre-trained target detection model to determine a stem region, removing the stem region in each slice image to obtain a stem-removed image corresponding to each slice image; combining all stem-removed images of the plant to obtain stem-removed three-dimensional data, performing corrosion processing on an ear grain region in the stem-removed three-dimensional data to obtain three-dimensional data after corrosion; performing inflation processing on the ear grain region in the three-dimensional data after corrosion to obtain ear grain mask data; and extracting an ear grain part from the three-dimensional CT data of the plant according to the ear grain mask data to obtain an ear grain three-dimensional image. The method can remove the influence of the stem and accurately extract the ear grain part of the plant.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a spike-grain extraction method and device based on three-dimensional CT data. Background Art

[0002] 3D CT scanning technology has been widely used in plant research due to its ability to non-destructively obtain high-resolution data on a plant's internal structure. This is particularly true for plants with spikes, where precise extraction of individual grains allows for better analysis of plant growth characteristics.

[0003] However, due to the high density of the stems of these plants with ears, the boundaries between ears and grains are blurred, which can easily lead to inaccurate extraction of ears and grains. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a spike-grain extraction method and device based on three-dimensional CT data to solve or partially solve the above technical problems.

[0005] Based on the above objectives, the present application provides a spike-grain extraction method based on three-dimensional CT data, comprising:

[0006] Acquiring plant three-dimensional CT data, and performing slicing processing on the plant three-dimensional CT data to obtain a plurality of slice images;

[0007] Using a pre-trained target detection model to perform recognition analysis on each of the slice images, determine the stem area, remove the stem area in each slice image, and obtain a stem-removed image corresponding to each of the slice images;

[0008] combining all the plant stem-removed images to obtain stem-removed three-dimensional data, and corroding the spikelet region in the stem-removed three-dimensional data to obtain corroded three-dimensional data;

[0009] performing expansion processing on the spikelet region in the eroded three-dimensional data to obtain spikelet mask data;

[0010] The spikelet portion is extracted from the plant three-dimensional CT data according to the spikelet mask data to obtain a spikelet three-dimensional image.

[0011] Based on the same inventive concept, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0012] From the above, it can be seen that the method and device for extracting grains based on three-dimensional CT data provided by the present application obtain plant three-dimensional CT data, which can accurately characterize the characteristics of the three-dimensional graphics of the plant. For better analysis, the plant three-dimensional CT data needs to be sliced, so that the plant three-dimensional CT data can be converted into two-dimensional slice images; then, the slice image is identified using the pre-trained target detection model to identify the stem area therein, so as to remove the stem area in the slice image, thereby eliminating the influence of the stem on grain extraction; then, all the stem-removed images are combined to form stem-removed three-dimensional data, so that the stem-removed three-dimensional data retains the grain area, but there will still be leaves, cobs, etc. Therefore, it is necessary to perform corrosion processing on the grain area in the three-dimensional data of the stalk removal so that the leaves, ear axes and other parts can be corroded to ensure that the grain part in the three-dimensional data after corrosion is more accurate. However, the grain part in the three-dimensional data after corrosion is relatively small and does not conform to the size of normal grains, so the three-dimensional data after corrosion needs to be expanded so that the grain mask data obtained is more accurate. Finally, the three-dimensional CT data of the plant is extracted using the grain mask data to obtain an accurate three-dimensional image of the grain. The process of obtaining the three-dimensional image of the grain can remove the influence of the stalk, so that the accuracy of the three-dimensional image of the grain is effectively improved, and better grain research and grain breeding are carried out based on the accurate three-dimensional image of the grain. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 This is a flow chart of the spike-grain extraction method based on three-dimensional CT data according to an embodiment of the present application;

[0015] Figure 2 A schematic diagram of three-dimensional data of stem removal according to an embodiment of the present application;

[0016] Figure 3 Schematic diagram of the corrosion treatment, expansion treatment and spike-grain extraction according to an embodiment of the present application;

[0017] Figure 4 This is a structural block diagram of a spike-grain extraction device based on three-dimensional CT data according to an embodiment of the present application;

[0018] Figure 5 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0020] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] Glossary:

[0022] CT: Computed Tomography.

[0023] YOLO: You Only Look Once, an object detection algorithm developed by Joseph Redmon et al., predicts bounding boxes and category probabilities directly from images through a single neural network, which makes it much faster than traditional two-stage object detection algorithms such as Faster R-CNN.

[0024] NMS: Non-Maximum Suppression, non-maximum suppression algorithm.

[0025] With the rapid development of intelligent agriculture, 3D CT scanning technology has been widely used in agricultural research due to its ability to non-destructively acquire high-resolution data on plant internal structures. In particular, in rice breeding, accurately extracting information about the morphology and structure of individual grains is crucial for yield assessment and variety optimization. However, the complex structure of plants (e.g., rice), particularly the interference from non-grain components such as the stem, leaves, and cob, poses numerous challenges to accurate grain extraction.

[0026] Traditional methods rely primarily on manual segmentation or simple image processing techniques, such as threshold segmentation and morphological operations. However, these methods often suffer from insufficient extraction accuracy or efficiency when processing complex 3D CT data due to issues such as the high density of the stalk's inner wall and the blurred boundaries between the kernel and the cob. Breakthroughs in object detection using deep learning techniques (such as the YOLO series of models) offer new possibilities for addressing these issues. However, these techniques still lack the full integration of the advantages of 3D data reconstruction and morphological processing, making it difficult to achieve efficient and accurate kernel extraction in complex scenarios.

[0027] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0028] The spike-grain extraction method based on 3D CT data proposed in the embodiments of the present application is mainly implemented on 3D CT data of various plants with spikes and grains, especially 3D CT data of cereal plants (such as rice, wheat or buckwheat, etc.).

[0029] like Figure 1 As shown, the method is applied to a data analysis device (such as a terminal device or a server), including:

[0030] Step 101 : Acquire plant three-dimensional CT data, and perform slicing processing on the plant three-dimensional CT data to obtain a plurality of slice images.

[0031] In a specific implementation, a CT device is used to perform a tomographic scan on the corresponding plant to obtain three-dimensional CT data of the plant, so that a data analysis device can obtain the three-dimensional CT data of the plant from the CT device.

[0032] In order to facilitate image analysis, it is necessary to slice the plant 3D CT data, thereby reducing the 3D data to multiple slice images of 2D data. The slicing method can be longitudinal slicing or transverse slicing, preferably transverse slicing.

[0033] Step 102: Use a pre-trained target detection model to perform recognition analysis on each of the slice images, determine the stem area, remove the stem area in each slice image, and obtain a stem-removed image corresponding to each of the slice images.

[0034] In a specific implementation, the target detection model is a YOLO model (eg, YOLOv11), which can identify the stem area in the image after pre-training. The target detection model can only perform recognition processing on two-dimensional images, so the slicing process of the above step 101 is required.

[0035] Step 103: combine all the plant stem removal images to obtain the stem removal three-dimensional data (e.g. Figure 2As shown in FIG), the spikelet region in the stalk-removed three-dimensional data is corroded to obtain the corroded three-dimensional data (as shown in FIG). Figure 3 shown).

[0036] During specific implementation, all stalk-removed images are combined in the corresponding position order to form stalk-removed three-dimensional data. The stalk-removed three-dimensional data retains the grain area, but it will still be affected by leaves, cobs and other parts. Therefore, it is necessary to use corrosion processing to remove image data other than the grains, and the grain area in the obtained corroded three-dimensional data is more accurate.

[0037] Step 104: dilate the spikelet region in the eroded three-dimensional data to obtain spikelet mask data (e.g. Figure 3 shown).

[0038] In specific implementation, since the spikelet region in the eroded three-dimensional data is relatively small and does not match the real spikelet, it is necessary to perform expansion processing on the spikelet region so that the final spikelet mask data will be more accurate.

[0039] Step 105: extract the spikelet portion from the plant 3D CT data according to the spikelet mask data to obtain a spikelet 3D image (e.g. Figure 3 shown).

[0040] During specific implementation, the grain mask data may be expanded, resulting in adjacent grains adhering to each other. The grains in the plant 3D CT data are not adhering to each other, and the grain mask data is a binary image that cannot reflect the grayscale information in the plant 3D CT data. Therefore, it is necessary to extract the area corresponding to the grain mask from the plant 3D CT data according to the grain mask data to obtain the grain 3D image. Specifically, the grain mask data can be subjected to intersection processing with the plant 3D CT data to obtain an accurate grain 3D image. Among them, the grain mask data is a binary image, so after the intersection processing with the plant 3D CT data, the portion of the plant 3D CT data that overlaps with the grain mask data can be extracted to obtain a relatively accurate grain 3D image.

[0041] In this way, the size, quantity, fullness and other analyses of the grains can be determined based on the three-dimensional images of the grains, providing an accurate basis for research on plant grains.

[0042] Through the above scheme, plant three-dimensional CT data is obtained, and the plant three-dimensional CT data can accurately characterize the three-dimensional morphological characteristics of the plant. For better analysis, the plant three-dimensional CT data needs to be sliced, so that the plant three-dimensional CT data can be converted into two-dimensional slice images; then, the slice image is identified using the pre-trained target detection model to identify the stem area therein, so as to remove the stem area in the slice image, thereby eliminating the influence of the stem on the grain extraction; then, all the stem-removed images are combined to form the stem-removed three-dimensional data, so that the grain area is retained in the stem-removed three-dimensional data, but there will still be the influence of leaves, cobs and other parts, so it is necessary to target the stem area. The grain area in the stalk-removed three-dimensional data is corroded so that the leaves, cobs and other parts can be corroded away, ensuring that the grain part in the corroded three-dimensional data is more accurate. However, the grain part in the corroded three-dimensional data is relatively small and does not conform to the size of a normal grain. Therefore, the corroded three-dimensional data needs to be expanded again, so that the grain mask data obtained is more accurate. Finally, the grain mask data is used to extract the plant three-dimensional CT data to obtain an accurate grain three-dimensional image. The process of obtaining the grain three-dimensional image can remove the influence of the stalk, so that the accuracy of the grain three-dimensional image is effectively improved, and better grain research and grain breeding are carried out based on the accurate grain three-dimensional image.

[0043] In some embodiments, step 101 includes:

[0044] Step 1011 : Acquire plant three-dimensional CT data, and perform artifact removal processing on the plant three-dimensional CT data to obtain artifact-removed three-dimensional data.

[0045] In practice, the original 3D plant CT data may contain artifacts that affect subsequent stem recognition, so it is necessary to first remove the artifacts from the 3D plant CT data. Specific removal methods can be filtering or binarization. This will result in clearer 3D data after artifact removal.

[0046] Step 1012: Slice the three-dimensional data after removing artifacts to obtain a plurality of two-dimensional slice images.

[0047] In specific implementation, since the target detection model processes two-dimensional images, it is necessary to slice the three-dimensional data after removing artifacts to obtain multiple two-dimensional slice images, and save these slice images in the form of grayscale images.

[0048] Through the above solution, the influence of artifacts in plant three-dimensional CT data can be removed, ensuring the clarity of the multiple two-dimensional slice images finally obtained.

[0049] In some embodiments, step 1011 includes:

[0050] Step 10111: Determine at least two thresholds corresponding to the plant three-dimensional CT data using a multi-threshold determination method.

[0051] In specific implementation, the multi-threshold determination method can use the multithresh function to calculate at least two thresholds, for example, select two thresholds , the formula is: T=multithresh(I,2)=(T1,T2), where T is the set of two thresholds, I is It belongs to the plant 3D CT data, and x, y, and z are the 3D coordinates of the 3D coordinate system constructed based on the plant 3D CT data.

[0052] The multithresh function is an image segmentation function in MATLAB, which is used to determine multiple thresholds for segmentation based on the image.

[0053] Step 10112: Select the highest threshold value from at least two threshold values ​​as the segmentation threshold value, and use the segmentation threshold value to perform binarization processing on the plant 3D CT data to obtain a binary mask.

[0054] In specific implementation, the highest threshold value will be selected from at least two threshold values. As the segmentation threshold, the plant 3D CT data is binarized according to the segmentation threshold to obtain a binary mask , which includes the white plant part with a pixel value of 1 and the black background part with a pixel value of 0.

[0055] For example, the specific formula is as follows:

[0056] .

[0057] Step 10113 : performing intersection processing on the binary mask and the plant three-dimensional CT data to obtain three-dimensional data after removing artifacts.

[0058] In the specific implementation, the binary mask is used to perform intersection processing (i.e., element-by-element multiplication processing) with the plant 3D CT data to obtain the 3D data after removing artifacts. , the formula is as follows:

[0059] .

[0060] Through the above solution, an accurate binary mask can be obtained, and then the binary mask can be used to remove artifacts in plant three-dimensional CT data, ensuring that the three-dimensional data after artifact removal is clearer and more accurate.

[0061] In some embodiments, step 1012 includes:

[0062] Step 10121 , slicing the three-dimensional data after artifact removal according to a predetermined slice scale or a predetermined number of slices to obtain a plurality of initial slice images.

[0063] In specific implementation, slicing can be performed according to a predetermined slicing scale or a predetermined number of slicing steps, so that P initial slicing images can be obtained. , where k is the sequence number, .

[0064]

[0065] .

[0066] in, is the pixel matrix of the initial slice image corresponding to sequence number k, and the size of the pixel matrix of the initial slice image is M×N (where M and N are both integers, M is the number of horizontal arrangements of the pixel matrix, and N is the number of vertical arrangements of the pixel matrix), is the matrix composed of each initial slice image.

[0067] Step 10122: grayscale processing is performed on the multiple initial slice images to obtain multiple two-dimensional slice images.

[0068] Through the above solution, it is ensured that the obtained initial slice image can accurately represent the appearance characteristics of the plant, making the slice image clearer and more accurate after grayscale processing.

[0069] In some embodiments, the object detection model training process includes:

[0070] Step A1: Obtain a sample slice image with a stem area marked.

[0071] In specific implementation, multiple sample plant 3D CT data are obtained, and each sample plant 3D CT data is sliced ​​to obtain multiple sample slice images corresponding to each sample plant 3D CT data to form a sample set, thereby obtaining multiple sample sets.

[0072] For each sample set, the stem region in each sample slice image in the sample set may be manually annotated.

[0073] Step A2: performing data amplification processing on the sample slice image to obtain an amplified sample slice image; wherein the data amplification processing includes: at least one of random scaling, random cropping or random rotation.

[0074] In specific implementation, since the number of manually annotated sample slice images is small, they need to be amplified. Each sample slice data can be amplified into multiple ones according to the corresponding data amplification processing method, thereby increasing the number of sample slice images.

[0075] Step A3: Use the amplified sample slice image to train the initial detection model, and use the back propagation method to determine the parameter update value according to the marked stem area, and use the parameter update value to adjust the parameters of the initial detection model to obtain a detection model after parameter adjustment.

[0076] In a specific implementation, an initial detection model is pre-built, and each amplified sample slice image is trained and processed to obtain an output result, which refers to the stem area in the amplified sample slice image. The amplified sample slice image refers to the above-mentioned sample slice data and the amplified sample slice image.

[0077] The output result is compared with the annotated stem area of ​​the amplified sample slice image to determine the corresponding loss function, calculate the loss value, and determine the adjustment gradient based on the loss value. The parameters of the initial model are then adjusted based on the adjustment gradient to obtain a detection model with adjusted parameters. For example, the specific formula is: ,in, represents the parameters of the initial model, is the learning rate, Indicates adjustment gradient.

[0078] Step A4: Determine whether the detection model after parameter adjustment meets the training end condition, and use the final detection model after parameter adjustment as the target detection model.

[0079] In a specific implementation, if all amplified sample slice images have been trained, or the accuracy of the detection model after parameter adjustment in identifying the stem region in the image exceeds a predetermined threshold, or the convergence value of the loss function corresponding to the detection model after parameter adjustment is less than a predetermined convergence threshold, the training end condition is determined to be met. The resulting detection model after parameter adjustment is then used as the target detection model, and the target detection model has the function of identifying the stem region in the image.

[0080] Through the above scheme, the number of sample slice images can be increased through amplification, and a large number of amplified sample slice images can be obtained, so that there are enough samples to train the initial detection model, ensuring that the target detection model obtained after training is more accurate.

[0081] In some embodiments, step 102 specifically includes, for each slice image:

[0082] Step 1021: Use a pre-trained target detection model to identify the stem target in the slice image and determine at least one anchor frame position of the stem target.

[0083] In a specific implementation, the target detection model has the ability to identify stem targets in an image, so that one or more anchor box positions can be determined for each stem target.

[0084] The anchor box position is represented as (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left corner of the anchor box rectangle, and (x2, y2) is the coordinate of the lower right corner of the anchor box rectangle.

[0085] Step 1022 : Using non-maximum suppression (NMS) on each anchor frame position, determine redundant anchor frame positions, remove the redundant anchor frame positions, and use the anchor frame after removing the redundancy as the stem anchor frame.

[0086] In specific implementation, if a stem target corresponds to multiple anchor box positions, and each anchor box position corresponds to a confidence level, it is necessary to use the non-maximum suppression method to process multiple anchor box positions. The process is as follows:

[0087] (1) Sort the multiple anchor box positions of a stem target according to the confidence level, select the anchor box position with the highest confidence level as the current selection box, and add it to the final detection result.

[0088] (2) Calculate the overlap between the remaining anchor box positions and the current selection box.

[0089] (3) If the overlap is greater than the set threshold, it is determined that the remaining anchor frame position has a high overlap with the current selection frame, indicating the same object, and the remaining anchor frame position will be removed as a redundant anchor frame position.

[0090] (4) Repeat the above steps (1) to (3) for the remaining anchor frame positions after removal until all anchor frame positions are processed. Finally, the accurate stem anchor frame of the stem target is obtained.

[0091] Step 1023: remove the area corresponding to the stem anchor frame in the slice image to obtain a stem-removed image corresponding to the slice image.

[0092] In specific implementation, the pixel value of the stem anchor frame can be set to 0 (black), so that the purpose of removing the stem anchor frame can be achieved.

[0093] Through the above scheme, the redundant positions of multiple anchor frames of each stem target can be accurately removed to avoid the overlapping anchor frame positions affecting the determination of the stem area, thereby obtaining an accurate stem anchor frame and ensuring the accuracy of removing the stem anchor frame from the slice image.

[0094] In some embodiments, the stalk-removed three-dimensional data is a three-dimensional binary image, and the three-dimensional binary image includes foreground pixels where the grain area is located and background pixels without grains.

[0095] The step 103 of performing corrosion processing on the spikelet region in the three-dimensional data of the stalks to obtain the three-dimensional data after corrosion includes:

[0096] Step 1031: construct a first structural element, and use the first structural element to traverse the stem removal three-dimensional data; wherein the first structural element is a three-dimensional structure.

[0097] In specific implementation, the first structural element It is a three-dimensional structure (for example, a 3×3×3 three-dimensional structure), and the shape and size of the three-dimensional structure can be defined according to actual needs.

[0098] Step 1032: During the traversal process, determine the first pixel that falls within the first structural element in the three-dimensional data after the stems are removed.

[0099] In a specific implementation, each pixel point in the three-dimensional data after the stem is removed is used as the central pixel point of the first structural element, and traversal is performed to determine the first falling element that falls within the first structural element.

[0100] Step 1033: In response to the first fallen-in pixels being all foreground pixels, the center pixel point where the first structural element is located is maintained as a foreground pixel; or, in response to the first fallen-in pixels being background pixels, the center pixel point where the first structural element is located is set as a background pixel.

[0101] In a specific implementation, only when all first falling elements within the first structuring element are foreground pixels will the corresponding central pixel be set as a foreground pixel (for example, set to 1 white). This can reduce the foreground portion and erode away the area of ​​the foreground portion that is smaller than the first structuring element.

[0102] The specific formula is: , where B is the 3D data after stem removal, with foreground pixels as 1 and background pixels as 0; Indicates that the first structural element is in pixels The collection after the center translation traversal; Represents structural elements At the pixel All non-zero elements of the position must fall completely within the plant 3D CT data in the foreground area.

[0103] Step 1034 , determining whether each pixel in the stem-removed three-dimensional data has been completely traversed by the first structuring element, and using the finally processed stem-removed three-dimensional data as the eroded three-dimensional data.

[0104] In a specific implementation, when each pixel in the three-dimensional data after the stems are removed is used as the central pixel point of the first structure element, it is determined that the traversal is completed, and the three-dimensional data after corrosion will be obtained.

[0105] Through the above scheme, the first structural element is used to traverse the three-dimensional data of stem removal and then realize the corrosion processing process, so that the edge of the foreground part becomes thinner, the details are reduced, and the smaller foreground details are removed, thereby avoiding the influence of the smaller foreground on the subsequent grain extraction.

[0106] In some embodiments, the eroded three-dimensional data includes: foreground pixels where the grain area is located and background pixels without grains;

[0107] Step 104 includes:

[0108] Step 1041 : construct a second structuring element, and use the second structuring element to traverse the eroded three-dimensional data; wherein the second structuring element is a three-dimensional structure.

[0109] In specific implementation, the second structural element It is a three-dimensional structure (for example, a 3×3×3 three-dimensional structure). The shape and size of the three-dimensional structure can be defined according to actual needs, and is preferably the same as the first structural element.

[0110] Step 1042: During the traversal process, determine a second falling pixel in the eroded three-dimensional data that falls within the second structure element.

[0111] In a specific implementation, each pixel point in the eroded three-dimensional data is used as the central pixel point of the second structure element and traversed, so that the second falling element falling into the second structure element can be determined.

[0112] Step 1043: In response to the presence of a foreground pixel in the second fallen-in pixels, the center pixel point of the second structural element is set as a foreground pixel; or, in response to the presence of all the second fallen-in pixels being background pixels, the center pixel point of the second structural element is set as a background pixel.

[0113] In a specific implementation, if there is a foreground pixel in the second falling element within the second structuring element, the corresponding center pixel is set as a foreground pixel (for example, set to 1 white). This can expand (dilerate) the foreground portion. Preferably, the second structuring element is the same as the first structuring element, so that the expanded size is similar to the grain area in the stalk-removed 3D data.

[0114] The specific formula is:

[0115] .

[0116] in, Represents the expanded spike-grain mask data; In pixels The collection after the center translation traversal; Represents the second structural element The non-zero part of has at least one pixel overlapping with the foreground area of ​​the eroded three-dimensional data.

[0117] Step 1044 , determining whether each pixel in the eroded three-dimensional data is completely traversed by the second structure element, and using the finally processed eroded three-dimensional data as the spike-grain mask data.

[0118] In a specific implementation, when each pixel in the eroded three-dimensional data is used as the central pixel point of the second structure element, it is determined that the traversal is completed, and the expanded spikelet mask data is obtained.

[0119] Through the above scheme, the second structural element can be used to expand the corroded three-dimensional data, expand the range of foreground pixels in the corroded three-dimensional data, make the grain part corresponding to the foreground pixel more matched with the size of the actual grain part, ensure that the obtained grain mask data is more consistent with the actual grain, and improve the accuracy of the grain mask data determination.

[0120] In some embodiments, step 105 includes:

[0121] Step 1051 : performing artifact processing on the plant three-dimensional CT data to obtain artifact-removed three-dimensional data.

[0122] Step 1052: retain the overlapping portion of the three-dimensional data after artifact removal and the spikelet mask data, remove the remaining portion, complete the extraction of the spikelet portion, and obtain the spikelet three-dimensional image.

[0123] In specific implementation, a logical AND operation is performed on the 3D image of the spikelet and the 3D data after removing artifacts to retain the part of the dilation result that overlaps with the foreground area of ​​the original image, thereby limiting the over-extension of the boundary that may be introduced by the dilation process and ensuring that the geometric shape and voxel information of the spikelet are the same as the original data. It can be expressed as:

[0124]

[0125] in, represents the three-dimensional data after removing artifacts, represents the three-dimensional image after removing artifacts, Represents the spike-grain mask data after expansion, which preserves the voxel intensity distribution characteristics in the original 3D image.

[0126] Through the above solution, it can be ensured that the obtained three-dimensional image of the ear and grain is more accurate.

[0127] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0128] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0129] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a spike-grain extraction device based on three-dimensional CT data.

[0130] refer to Figure 4 , the device comprises:

[0131] The slice processing module 201 is configured to acquire plant 3D CT data and perform slice processing on the plant 3D CT data to obtain a plurality of slice images;

[0132] The stem removal module 202 is configured to use a pre-trained target detection model to perform recognition analysis on each of the slice images, determine the stem area, remove the stem area from each slice image, and obtain a stem-removed image corresponding to each slice image;

[0133] an erosion processing module 203 configured to combine all the stalk-removed images of the plant to obtain stalk-removed three-dimensional data, and perform an erosion process on the spikelet region in the stalk-removed three-dimensional data to obtain eroded three-dimensional data;

[0134] The expansion processing module 204 is configured to perform expansion processing on the spikelet region in the eroded three-dimensional data to obtain spikelet mask data;

[0135] The spikelet extraction module 205 is configured to extract the spikelet portion from the plant three-dimensional CT data according to the spikelet mask data to obtain a spikelet three-dimensional image.

[0136] In some embodiments, the slice processing module 201 includes:

[0137] an artifact removal unit configured to acquire plant three-dimensional CT data, perform artifact removal processing on the plant three-dimensional CT data, and obtain artifact-removed three-dimensional data;

[0138] The slicing processing unit is configured to perform slicing processing on the three-dimensional data after artifact removal to obtain a plurality of two-dimensional slice images.

[0139] In some embodiments, the artifact removal unit is specifically configured to:

[0140] Determining at least two thresholds corresponding to the plant three-dimensional CT data using a multi-threshold determination method;

[0141] Selecting a highest threshold value from at least two threshold values ​​as a segmentation threshold value, and performing binarization processing on the plant three-dimensional CT data using the segmentation threshold value to obtain a binary mask;

[0142] Intersection processing is performed on the binary mask and the plant three-dimensional CT data to obtain three-dimensional data after artifacts are removed.

[0143] In some embodiments, the slice processing unit is specifically configured to:

[0144] Slicing the artifact-removed three-dimensional data according to a predetermined slice scale or a predetermined number of slices to obtain a plurality of initial slice images;

[0145] Grayscale processing is performed on the multiple initial slice images to obtain multiple two-dimensional slice images.

[0146] In some embodiments, the apparatus further comprises a model training module configured to:

[0147] Acquire a sample slice image with the stem area marked;

[0148] Performing data amplification processing on the sample slice image to obtain an amplified sample slice image; wherein the data amplification processing includes: at least one of random scaling, random cropping or random rotation;

[0149] The initial detection model is trained using the amplified sample slice image, and parameter update values ​​are determined according to the marked stem area using a back propagation method, and the parameter update values ​​are used to adjust the parameters of the initial detection model to obtain a detection model after parameter adjustment;

[0150] It is determined that the detection model after parameter adjustment meets the training end condition, and the final detection model after parameter adjustment is used as the target detection model.

[0151] In some embodiments, the stem removal module 202 is specifically configured to:

[0152] For each of the slice images:

[0153] Using a pre-trained target detection model to identify a stem target in the slice image, and determining at least one anchor frame position of the stem target;

[0154] Using the non-maximum suppression method for each anchor frame position to determine the redundant anchor frame position, removing the redundant anchor frame position, and using the anchor frame after removing the redundancy as the stem anchor frame;

[0155] The area corresponding to the stem anchor frame in the slice image is removed to obtain a stem-removed image corresponding to the slice image.

[0156] In some embodiments, the stalk-removed three-dimensional data is a three-dimensional binary image, wherein the three-dimensional binary image includes foreground pixels where the grain area is located and background pixels without grains;

[0157] The corrosion processing module 203 is specifically configured to:

[0158] Constructing a first structural element, and using the first structural element to traverse the stem removal three-dimensional data; wherein the first structural element is a three-dimensional structure;

[0159] During the traversal process, determining a first pixel falling within a first structural element in the three-dimensional data of the stem removal;

[0160] In response to all the first fallen-in pixels being foreground pixels, the center pixel point where the first structuring element is located is maintained as a foreground pixel; or, in response to the presence of background pixels around the first fallen-in pixels, the center pixel point where the first structuring element is located is set as a background pixel;

[0161] It is determined that each pixel in the stem-removed three-dimensional data is completely traversed by the first structure element, and the finally processed stem-removed three-dimensional data is used as the corroded three-dimensional data.

[0162] In some embodiments, the eroded three-dimensional data includes: foreground pixels where the grain area is located and background pixels without grains;

[0163] The expansion processing module 204 is specifically configured to:

[0164] Constructing a second structuring element, and traversing the eroded three-dimensional data using the second structuring element; wherein the second structuring element is a three-dimensional structure;

[0165] During the traversal process, determining a second falling pixel in the eroded three-dimensional data that falls within the second structure element;

[0166] In response to a foreground pixel being present among the second fallen-in pixels, setting the center pixel point where the second structuring element is located as a foreground pixel; or, in response to all the second fallen-in pixels being background pixels, setting the center pixel point where the second structuring element is located as a background pixel;

[0167] It is determined that each pixel in the corroded three-dimensional data is completely traversed by the second structure element, and the finally processed corroded three-dimensional data is used as the spike-grain mask data.

[0168] In some embodiments, the spike-grain extraction module 205 is specifically configured to:

[0169] performing artifact processing on the plant three-dimensional CT data to obtain artifact-removed three-dimensional data;

[0170] The overlapping part of the three-dimensional data after removing the artifacts and the ear-grain mask data is retained, and the remaining part is removed to complete the extraction of the ear-grain part and obtain the three-dimensional image of the ear-grain.

[0171] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0172] The apparatus of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0173] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the computer program.

[0174] Figure 5 A more specific hardware structure diagram of an electronic device provided in this embodiment is shown. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0175] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0176] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0177] The input / output interface 1030 is used to connect to input / output modules to enable information input and output. The input / output modules can be configured as components within the device (not shown) or externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, and various sensors. Output devices may include a display, speaker, vibrator, indicator light, and the like.

[0178] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.).

[0179] The bus 1050 comprises a pathway for transmitting information between various components of the device, such as the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 .

[0180] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0181] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0182] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0183] The computer-readable media of this embodiment includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0184] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0185] Based on the same concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0186] It is understandable that before using the technical solutions of each embodiment of this application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0187] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of this application based on the prompt message.

[0188] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0189] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0190] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0191] In addition, to simplify the description and discussion, and to avoid obscuring the understanding of the embodiments of the present application, well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. Furthermore, devices may be shown in block diagram form to avoid obscuring the understanding of the embodiments of the present application, and this also takes into account the fact that the implementation details of these block diagram devices are highly dependent on the platform on which the embodiments of the present application will be implemented (i.e., these details should be fully understood by those skilled in the art). Where specific details (e.g., circuits) are set forth to describe the exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations therefrom. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0192] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0193] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A spike-grain extraction method based on three-dimensional CT data, characterized in that: include: Determining at least two thresholds corresponding to the plant three-dimensional CT data using a multi-threshold determination method; Selecting a highest threshold value from at least two threshold values ​​as a segmentation threshold value, and performing binarization processing on the plant three-dimensional CT data using the segmentation threshold value to obtain a binary mask; Performing intersection processing on the binary mask and the plant three-dimensional CT data to obtain three-dimensional data after removing artifacts; Slicing the three-dimensional data after artifact removal to obtain a plurality of two-dimensional slice images; Using a pre-trained target detection model to perform recognition analysis on each of the slice images, determine the stem area, remove the stem area in each slice image, and obtain a stem-removed image corresponding to each of the slice images; combining all the plant stem-removed images to obtain stem-removed three-dimensional data, and corroding the spikelet region in the stem-removed three-dimensional data to obtain corroded three-dimensional data; performing expansion processing on the spikelet region in the eroded three-dimensional data to obtain spikelet mask data; The spikelet portion is extracted from the plant three-dimensional CT data according to the spikelet mask data to obtain a spikelet three-dimensional image.

2. The method according to claim 1, characterized in that The slicing process is performed on the three-dimensional data after removing artifacts to obtain a plurality of two-dimensional slice images, including: Slicing the artifact-removed three-dimensional data according to a predetermined slice scale or a predetermined number of slices to obtain a plurality of initial slice images; Grayscale processing is performed on the multiple initial slice images to obtain multiple two-dimensional slice images.

3. The method according to claim 1, characterized in that The training process of the target detection model includes: Acquire a sample slice image with the stem area marked; Performing data amplification processing on the sample slice image to obtain an amplified sample slice image; wherein the data amplification processing includes: at least one of random scaling, random cropping or random rotation; The initial detection model is trained using the amplified sample slice image, and parameter update values ​​are determined according to the marked stem area using a back propagation method, and the parameter update values ​​are used to adjust the parameters of the initial detection model to obtain a detection model after parameter adjustment; It is determined that the detection model after parameter adjustment meets the training end condition, and the final detection model after parameter adjustment is used as the target detection model.

4. The method according to claim 1, wherein The method of using a pre-trained target detection model to perform recognition analysis on each slice image, determining a stem region, and removing the stem region from each slice image to obtain a stem-removed image corresponding to each slice image includes: For each of the slice images: Using a pre-trained target detection model to identify a stem target in the slice image, and determining at least one anchor frame position of the stem target; Using the non-maximum suppression method for each anchor frame position to determine the redundant anchor frame position, removing the redundant anchor frame position, and using the anchor frame after removing the redundancy as the stem anchor frame; The area corresponding to the stem anchor frame in the slice image is removed to obtain a stem-removed image corresponding to the slice image.

5. The method according to claim 1, wherein The stalk-removed three-dimensional data is a three-dimensional binary image, and the three-dimensional binary image includes foreground pixels where the grain area is located and background pixels without grains; The corroding the grain region of the stalk-removed three-dimensional data to obtain the corroded three-dimensional data includes: Constructing a first structural element, and using the first structural element to traverse the stem removal three-dimensional data; wherein the first structural element is a three-dimensional structure; During the traversal process, determining a first pixel falling within a first structural element in the three-dimensional data of the stem removal; In response to all the first fallen-in pixels being foreground pixels, the center pixel point where the first structuring element is located is maintained as a foreground pixel; or, in response to the presence of background pixels around the first fallen-in pixels, the center pixel point where the first structuring element is located is set as a background pixel; It is determined that each pixel in the stem-removed three-dimensional data is completely traversed by the first structure element, and the finally processed stem-removed three-dimensional data is used as the corroded three-dimensional data.

6. The method according to claim 1, characterized in that The three-dimensional data after the corrosion includes: foreground pixels where the grain area is located and background pixels without grains; The step of performing expansion processing on the spikelet region in the eroded three-dimensional data to obtain spikelet mask data includes: Constructing a second structuring element, and traversing the eroded three-dimensional data using the second structuring element; wherein the second structuring element is a three-dimensional structure; During the traversal process, determining a second falling pixel in the eroded three-dimensional data that falls within the second structure element; In response to a foreground pixel being present among the second fallen-in pixels, setting the center pixel point where the second structuring element is located as a foreground pixel; or, in response to all the second fallen-in pixels being background pixels, setting the center pixel point where the second structuring element is located as a background pixel; It is determined that each pixel in the corroded three-dimensional data is completely traversed by the second structure element, and the finally processed corroded three-dimensional data is used as the spike-grain mask data.

7. The method according to claim 1, characterized in that The step of extracting the spikelet portion from the plant three-dimensional CT data according to the spikelet mask data to obtain a spikelet three-dimensional image comprises: performing artifact processing on the plant three-dimensional CT data to obtain artifact-removed three-dimensional data; The overlapping part of the three-dimensional data after removing the artifacts and the ear-grain mask data is retained, and the remaining part is removed to complete the extraction of the ear-grain part and obtain the three-dimensional image of the ear-grain.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images