A deep learning-based high-throughput visual analysis method and system for corn kernel and embryo volume

Abstract

The present application relates to the technical field of image data processing, and discloses a corn kernel and embryo volume high-throughput visual analysis method and system based on deep learning, which can obtain at least one folder according to a file input path set by a user, and each folder includes a gray scale sequence obtained by computer tomography (CT) slicing of corn kernel groups in a horizontal direction. Based on a parallel pipeline processing mode, a set batch size and a deep learning segmentation model, the gray scale sequence in the folder is preprocessed and image segmented to obtain kernel binary segmentation mask sequences and embryo binary segmentation mask sequences, and then the kernel quantity and the volume related data of the kernel and the embryo are determined, and the related images and data are visually output. The present application can complete the processing of the high-throughput data of the CT slice sequence of the corn kernel, and the nondestructive measurement and visual display of the volume related data of the kernel and the embryo, and effectively improve the measurement efficiency.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a high-throughput visualization analysis method and system for corn kernel and germ volume based on deep learning. Background Technology

[0002] With the development of science and technology, the cross-application technology of smart agriculture and computer vision is constantly improving.

[0003] In maize breeding, the volume correlation between kernel and embryo is a key phenotypic trait for evaluating maize quality (such as oil content and nutritional value). Therefore, technicians need to calculate the volume correlation between kernel and embryo. Specifically, the displacement method is generally used to measure the volume correlation between kernel and embryo.

[0004] However, the drainage method is inefficient for calculating data related to grain and embryo volume, and it can damage the grain, making subsequent breeding impossible. Summary of the Invention

[0005] This invention provides a high-throughput visualization analysis method and system for maize kernel and embryo volume based on deep learning. This addresses the shortcomings of the drainage method in related technologies, which suffers from low efficiency in calculating kernel and embryo volume data and damages the kernels, hindering subsequent breeding. Based on computer vision technology, a parallel pipeline processing mode, a deep learning segmentation model, and a newly created spatiotemporal consistency filtering algorithm, the system processes high-throughput data from maize kernel computed tomography (CT) slice sequences and performs non-destructive calculations of kernel and embryo volume data. It enables image segmentation, visualization of kernel and embryo volume data, and effectively improves calculation efficiency.

[0006] In a first aspect, the present invention provides a high-throughput visualization analysis method for maize kernel and germ volume based on deep learning, comprising: At least one folder is obtained according to the file input path set by the user, and each folder includes a grayscale image sequence obtained by CT slices of corn kernels in the horizontal direction; For any of the folders, based on the parallel pipeline processing mode, the set batch size and the deep learning segmentation model, the grayscale image sequence in the folder is preprocessed and segmented to obtain the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence. The number of seeds is counted based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence; and, based on the number of seeds, the interpolation integration algorithm, the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence, the volume-related data of seeds and embryos are determined. The grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data are visualized and output.

[0007] Optionally, the step of counting the number of seeds based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence includes: Based on the spatiotemporal consistency filtering algorithm, each seed binary segmentation mask in the seed binary segmentation mask sequence is filtered to obtain the corresponding filtered mask sequence. Based on the image order in the filtered mask sequence, each mask in the filtered mask sequence is stacked to obtain a three-dimensional voxel matrix; The number of seeds is obtained by counting the seeds in the three-dimensional voxel matrix using a three-dimensional connected component labeling algorithm.

[0008] Optionally, the step of filtering each seed binary segmentation mask in the seed binary segmentation mask sequence based on the spatiotemporal consistency filtering algorithm to obtain the corresponding filtered mask sequence includes: For any seed binary segmentation mask in the seed binary segmentation mask sequence, obtain k masks before the seed binary segmentation mask, the seed binary segmentation mask, and k masks after the seed binary segmentation mask from the seed binary segmentation mask sequence. Arrange each obtained mask according to the order of mask sequence to obtain a corresponding local mask sequence. Perform a logical AND operation on each mask in the local mask sequence to determine the overlapping region mask of the local mask sequence. Replace the seed binary segmentation mask in the seed binary segmentation mask sequence with the overlapping region mask to achieve filtering of the seed binary segmentation mask. The mask sequence obtained by filtering each of the binary segmentation masks of the seeds is determined as the filtered mask sequence.

[0009] Optionally, the volume-related data includes total grain volume, total embryo volume, average grain volume, average embryo volume, and volume ratio, wherein the volume ratio is the ratio of the average grain volume to the average embryo volume.

[0010] Optionally, determining the volume-related data of the grains and embryos based on the grain quantity, interpolation integration algorithm, grain binary segmentation mask sequence, and embryo binary segmentation mask sequence includes: According to the interpolation integration algorithm, the volume of each grain binary segmentation mask in the grain binary segmentation mask sequence and the volume of each embryo binary segmentation mask in the embryo binary segmentation mask sequence are calculated to obtain the total grain volume and the total embryo volume. The average grain volume and the average embryo volume are obtained by dividing the total grain volume and the total embryo volume by the number of grains, respectively. The volume ratio is calculated based on the average volume of the grain and the average volume of the embryo. The total volume of the grains, the total volume of the embryos, the average volume of the grains, the average volume of the embryos, and the volume ratio are collectively used as the volume-related data.

[0011] Optionally, the preprocessing and image segmentation of the grayscale image sequence in the folder based on the parallel pipeline processing mode, the set batch size, and the deep learning segmentation model to obtain the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence includes: Based on the parallel pipeline processing mode and the set batch size, the deep learning segmentation model is invoked to preprocess the grayscale image sequence in the folder, perform dual-channel parallel segmentation of seeds and embryos, and post-processing to obtain the corresponding seed binary segmentation mask sequence, embryo binary segmentation mask sequence, and color segmentation mask sequence. Figure 2 Value segmentation mask sequence; The method further includes: Based on the file output path set by the user, the seed binary segmentation mask sequence, the embryo binary segmentation mask sequence, and the color... Figure 2 The value segmentation mask sequence is saved to the corresponding target storage space.

[0012] Optionally, the visualization output of the grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data includes: Based on the grayscale image sequence, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence, respectively, generate and output the corresponding three-dimensional models; Based on the seed quantity and volume-related data, a statistical chart recording the seed quantity and volume-related data is generated and output.

[0013] Optionally, after obtaining at least one folder based on the file input path set by the user, the method further includes: Read the dots per inch (DPI) from the image metadata of the grayscale image sequence in the folder, and convert the DPI into the corresponding physical pixel size; Provide the user with a manual overlay image size option, receive the pixel side length and slice spacing input by the user when the image DPI is missing or inconsistent, and use the pixel side length and slice spacing input by the user to perform volume calculation; Create a global log, which is used to record the processing progress of each folder, abnormal DPI settings, and / or file corruption information; When processing of the folder fails, the global log records the failure information, interrupts the processing of the folder, and continues to process the next folder.

[0014] Optionally, after obtaining at least one folder based on the file input path set by the user, the method further includes: For any of the folders, if the folder contains both images and subfolders, or if the folder contains neither images nor subfolders, then it is determined to be invalid input; When the folder is determined to be invalid input or other preset errors occur, the created global log records the failure information, reports the error through the interface, and at least terminates the processing of the folder.

[0015] Secondly, the present invention provides a high-throughput visualization analysis system for maize kernel and germ volume based on deep learning, comprising: The acquisition unit is used to acquire at least one folder according to the file input path set by the user, and each folder includes a grayscale image sequence obtained by performing a computed tomography CT slice of corn kernels along the horizontal direction. The segmentation unit is used to preprocess and segment the grayscale image sequence in any folder based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, so as to obtain a seed binary segmentation mask sequence and an embryo binary segmentation mask sequence. A counting unit is used to count the number of seeds based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence. The determining unit is used to determine the volume-related data of the seeds and embryos based on the seed quantity, the interpolation integration algorithm, the seed binary segmentation mask sequence, and the embryo binary segmentation mask sequence. The output unit is used to visualize and output the grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data.

[0016] The present invention provides a high-throughput visualization analysis method and system for maize kernel and embryo volume based on deep learning. It can obtain at least one folder according to a user-defined file input path, each folder containing a sequence of grayscale images obtained from CT slices of maize kernels along a horizontal direction. For any folder, based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, the grayscale image sequences in the folder are preprocessed and segmented to obtain a kernel binary segmentation mask sequence and an embryo binary segmentation mask sequence. The number of kernels is counted based on a created spatiotemporal consistency filtering algorithm and the kernel binary segmentation mask sequence. Furthermore, based on the kernel number, interpolation integration algorithm, kernel binary segmentation mask sequence, and embryo binary segmentation mask sequence, volume-related data of kernels and embryos are determined. Finally, the grayscale image sequence, kernel binary segmentation mask sequence, embryo binary segmentation mask sequence, kernel number, and / or volume-related data are visualized and output. This invention can process high-throughput data of corn kernel CT slice sequences and perform non-destructive calculations of kernel and embryo volume-related data based on computer vision technology, parallel pipeline processing mode, deep learning segmentation model, and created spatiotemporal consistency filtering algorithm. It can achieve image segmentation, visualization of kernel and embryo volume-related data, and effectively improve calculation efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a high-throughput visualization analysis method for corn kernel and germ volume based on deep learning, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating how a user sets file input and output paths on a graphical user interface, as provided in an embodiment of the present invention. Figure 3 This invention provides a progress display diagram for processing a grayscale image sequence in a folder. Figure 4 A schematic diagram of a grayscale image, a grain binary segmentation mask, and an embryo binary segmentation mask provided for embodiments of the present invention; Figure 5 A schematic diagram illustrating the storage of a grain binary segmentation mask and an embryo binary segmentation mask in a folder, as provided in an embodiment of the present invention; Figure 6 A schematic diagram of a target storage space for storing output data provided in an embodiment of the present invention; Figure 7 This invention provides a table for storing data related to seed and embryo volume. Figure 8 This is a schematic diagram illustrating the content recorded in a global log according to an embodiment of the present invention; Figure 9 A schematic diagram of the structure of a high-throughput visualization analysis system for corn kernel and germ volume based on deep learning, provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] The relevant technologies have the following limitations in calculating the volume correlation data of corn kernels and embryos: 1. The measurement methods of related technologies are inefficient and damage maize seeds: the volume of the internal embryo cannot be measured non-destructively by relying on the water displacement method, and subsequent breeding cannot be carried out because the sample needs to be destroyed.

[0021] 2. Difficulty in algorithm implementation: Although some studies have used computed tomography (CT) combined with deep learning algorithms for segmentation, most of them remain at the theoretical verification stage and are usually composed of scattered Python scripts, lacking a unified interactive interface.

[0022] 3. High operational threshold: Existing methods usually require breeding experts to have programming skills to modify parameters and convert data formats, and require manual calculation of the correspondence between pixels and physical size.

[0023] 4. Lack of high-throughput processing capability: Existing code is mostly single-threaded, single-seed serial processing (image reading -> prediction -> saving), which results in low memory utilization and severe input / output (IO) blocking when processing GB-level CT slice sets, failing to meet the needs of large-scale breeding population analysis.

[0024] 5. Image adhesion and counting challenges caused by dense clustering: In high-throughput Micro-CT scanning scenarios, corn kernels are often randomly placed and densely packed. This leads to the edges of adjacent kernels easily contacting each other in two-dimensional slice images (XY plane), forming pseudo-adhesion. Morphological erosion or opening operations of related techniques are ineffective and inefficient in dealing with such complex situations.

[0025] 6. Lack of fault tolerance mechanism: When faced with large-scale file input, there is a lack of automatic processing and logging mechanisms for abnormal images, missing dots per inch settings, etc., which makes the program prone to interruption.

[0026] The following is combined Figures 1-8 This invention describes a deep learning-based high-throughput visualization analysis method for maize kernel and germ volume.

[0027] like Figure 1 As shown in the figure, this embodiment proposes a first method for high-throughput visualization analysis of corn kernel and germ volume based on deep learning. This method may include the following steps: S101. Obtain at least one folder according to the file input path set by the user, and each folder contains a grayscale image sequence obtained by CT slices of corn kernels in the horizontal direction.

[0028] It should be noted that this embodiment can be applied to electronic devices such as computers, laptops, and mobile phones.

[0029] The sample seed groups corresponding to different folders can be different. Optionally, each folder has a unique corresponding sample seed group.

[0030] Specifically, the sample kernel group may include multiple corn kernels. In this embodiment, all kernels in the sample kernel group can be placed on a flat table or laboratory bench. Using equipment such as micro-CT, scanning electron microscope, or digital slide scanner, the sample kernel group is sliced ​​and scanned along the horizontal direction of the table or laboratory bench where the sample kernel group is placed to obtain a grayscale image sequence.

[0031] The grayscale image sequence includes multiple grayscale images, each obtained by scanning a slice of the sample seed group in the vertical direction. It can be understood that each pixel in the grayscale image represents a grayscale value indicating brightness.

[0032] Specifically, users can store the grayscale image sequence of each sample seed group in a unique corresponding folder. The electronic device in this embodiment can have built-in intelligent parameter configuration logic, allowing users to configure relevant parameters according to actual needs.

[0033] like Figure 2As shown, users can set the file input and output paths on the graphical user interface. This embodiment can retrieve each folder stored in the corresponding storage space based on the user-defined file input path. It should be noted that the output path is the path used to store intermediate processing data and result data.

[0034] Optionally, users can also set image parameters and related model parameters of the model to be called through a graphical user interface. This embodiment can implement an automatic dots per inch (DPI) parsing mechanism. Specifically, this embodiment can automatically read the DPI information in the input image metadata and automatically convert it into physical pixel size, without requiring manual calculation by the user; it also provides a manual overwrite option, which can read the physical pixel size information input by the user to adapt to special cases of DPI loss.

[0035] Optionally, this embodiment can support dual-mode input logic, allowing the user to select either single-sample mode or high-throughput mode. In single-sample mode, the user only needs to input a single folder. In high-throughput mode, the user can set a parent folder containing multiple subfolders, each subfolder containing a sequence of grayscale images. This embodiment can automatically identify and switch between single-sample mode and high-throughput mode based on the user-defined input path.

[0036] Optionally, in other deep learning-based high-throughput visualization analysis methods for maize kernel and embryo volume proposed in this embodiment, the method may further include the following after step S101: For any folder, if the folder contains both images and subfolders, or if the folder contains neither images nor subfolders, then it is considered invalid input; When the folder is determined to be invalid input or other preset errors occur, the created global log records the failure information, reports the error through the interface, and at least terminates the processing of the folder.

[0037] Specifically, if a folder contains only multiple images or only multiple subfolders, it can be considered valid input, and the folder can continue to be processed.

[0038] It should be noted that if the input path folder contains image files (png, .jpg, .jpeg) and has no subfolders, all images within it will be processed directly (single folder mode). If the input path folder consists entirely of subfolders, each subfolder will be traversed, and a list of image files within each subfolder will be retrieved (only the above formats are supported). For each subfolder, if the image list is not empty, processing will proceed; otherwise, it will be skipped and logged, without affecting the processing of subsequent subfolders (parent folder mode). If the input path folder contains both image files and subfolders, or neither, it will be considered invalid input, an error message will be displayed, and processing will terminate. Files in non-image formats will be automatically ignored when retrieving the image list and will not cause errors; however, if a folder does not contain any supported image files, that folder will be skipped.

[0039] S102. For any folder, based on the parallel pipeline processing mode, the set batch size and the deep learning segmentation model, the grayscale image sequence in the folder is preprocessed and segmented to obtain the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence.

[0040] Specifically, the electronic device in this embodiment can be configured with a high-throughput parallel pipeline architecture based on a producer-consumer model. In this embodiment, for any folder, based on the parallel pipeline processing mode corresponding to this architecture, the set batch size, and the image segmentation model obtained through deep learning, the grayscale image sequence of the folder can be preprocessed and segmented to perform dual-channel parallel segmentation of each grayscale image in the grayscale image sequence, yielding a seed binary segmentation mask sequence and an embryo binary segmentation mask sequence.

[0041] Wherein, batch size refers to the number of images processed in each batch during the parallel pipeline processing in this embodiment.

[0042] Specifically, in the preprocessing of the grayscale image in this embodiment, image reading and color space conversion can be performed to obtain the converted image, and the corresponding number of converted images can be packaged according to the batch size. Among them, the color space conversion can include converting the image color format from Blue-Green-Red (BGR) to Red-Green-Blue (RGB).

[0043] Optionally, in other deep learning-based high-throughput visualization analysis methods for maize kernel and embryo volume proposed in this embodiment, step S102 includes: Based on a parallel pipeline processing mode and a set batch size, a deep learning segmentation model is invoked to preprocess the grayscale image sequences in the folder, perform dual-channel parallel segmentation of seeds and embryos, and post-processing to obtain the corresponding seed binary segmentation mask sequence, embryo binary segmentation mask sequence, and color segmentation mask sequence. Figure 2 Value segmentation mask sequence.

[0044] The method may also include: Based on the user-defined file output path, the seed binary segmentation mask sequence, embryo binary segmentation mask sequence, and color sequence will be output. Figure 2 The value segmentation mask sequence is saved to the corresponding target storage space.

[0045] Optionally, in this embodiment, after preprocessing and image segmentation of the batch of grayscale images during parallel pipeline processing, post-processing can continue. During post-processing, this embodiment can perform mask extraction, binarization, pseudo-color image generation, and file writing on the seed binary segmentation mask. This embodiment can also call a pre-trained image segmentation model, such as the U-Net model, for image segmentation.

[0046] It should be noted that this embodiment is based on a parallel pipelined processing method, which can maximize the utilization of the graphics processing unit (GPU). While the GPU is performing calculations, the central processing unit (CPU) processes input / output (I / O) and post-processing in parallel, which significantly improves throughput.

[0047] like Figure 3 As shown, this embodiment can also record and display the processing progress of grayscale image sequences in each folder.

[0048] like Figure 4 The grayscale image, grain binary segmentation mask, and embryo binary segmentation mask shown in this embodiment can identify and mark the grain and embryo in the grayscale image.

[0049] like Figure 5 As shown, in this embodiment, all binary segmentation masks for the same sample grain group and binary segmentation masks for the embryo can be stored in the same folder. Figure 5 In Chinese, "embryos" means embryo, "grains" means grain, and "slice" means slice.

[0050] like Figure 6 As shown, this embodiment can store the folder in the target storage space, such as... Figure 5 The folders in the folder are named 863, 1511, 864, 1552, 865, 1210, 1356-12kernel, and 1510.

[0051] S103. The number of seeds is counted based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence.

[0052] Specifically, in this embodiment, for any folder, the created spatiotemporal consistency filtering algorithm can be used to process the seed binary segmentation mask sequence corresponding to the folder in order to determine the number of seeds in the sample seed group corresponding to the folder.

[0053] Optionally, step S103 includes: Based on the spatiotemporal consistency filtering algorithm, each grain binary segmentation mask in the grain binary segmentation mask sequence is filtered to obtain the corresponding filtered mask sequence. Based on the image order in the filtered mask sequence, each mask in the filtered mask sequence is stacked to obtain a three-dimensional voxel matrix; The number of seeds is obtained by counting the seeds in a three-dimensional voxel matrix using a three-dimensional connected component labeling algorithm.

[0054] Optionally, the above-mentioned spatiotemporal consistency filtering algorithm filters each grain binary segmentation mask in the grain binary segmentation mask sequence to obtain the corresponding filtered mask sequence, including: For any seed binary segmentation mask in the seed binary segmentation mask sequence, obtain the k masks before the seed binary segmentation mask, the seed binary segmentation mask, and the k masks after the seed binary segmentation mask from the seed binary segmentation mask sequence. Arrange each obtained mask according to the order of the masks to obtain the corresponding local mask sequence. Perform a logical AND operation on each mask in the local mask sequence to determine the overlapping region mask of the local mask sequence. Replace the seed binary segmentation mask in the seed binary segmentation mask sequence with the overlapping region mask to achieve filtering of the seed binary segmentation mask. The mask sequence obtained after filtering each grain binary segmentation mask is determined as the filtered mask sequence.

[0055] Specifically, this embodiment employs an anti-adhesion counting strategy based on the intersection of Z-axis sliding windows for kernel counting. This embodiment addresses the problem of pseudo-adhesion in three-dimensional space caused by the close arrangement of corn kernels in high-throughput CT slices, and designs a spatiotemporal consistency filtering algorithm for processing. This embodiment can define a Z-axis sliding window WindowSize, with a default value of 21 layers. For each kernel binary segmentation mask S in the kernel binary segmentation mask sequence... i The local sequence W is formed by extracting adjacent slices before and after it. i ={S i-k ,..., S i ,...,S i+k}, for the local sequence W i Perform a logical AND operation on all binary masks within the window to calculate the overlapping region mask within that window, and then replace the original slice S with this overlapping region mask. iThis embodiment executes a spatiotemporal consistency filtering algorithm, which is equivalent to performing morphological erosion in the Z-axis direction of the sample seed group. This can effectively cut off discontinuous and minute contact points (adhesion bridges) between two seeds, while filtering out random noise points that do not have spatial continuity, thus achieving strong adhesion segmentation.

[0056] S104. Based on the number of grains, the interpolation integration algorithm, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence, determine the volume-related data of the grains and embryos.

[0057] Specifically, in this embodiment, after determining the number of seeds and the seed binary segmentation mask sequence of the sample seed group, the volume-related data of seeds and embryos can be determined based on the interpolation integration algorithm, the seed binary segmentation mask sequence, the embryo binary segmentation mask sequence, and the number of seeds.

[0058] Optionally, the volume-related data includes total grain volume, total embryo volume, average grain volume, average embryo volume, and volume ratio, where the volume ratio is the ratio of average grain volume to average embryo volume.

[0059] Optionally, step S104 includes: Based on the interpolation integration algorithm, the volume of each grain binary segmentation mask in the grain binary segmentation mask sequence and the volume of each embryo binary segmentation mask in the embryo binary segmentation mask sequence are calculated to obtain the total grain volume and the total embryo volume. Divide the total volume of grains and the total volume of embryos by the number of grains to obtain the average volume of grains and the average volume of embryos, respectively. Calculate the volume ratio based on the average volume of the grain and the average volume of the embryo; The total volume of grains, the total volume of embryos, the average volume of grains, the average volume of embryos, and the volume ratio are all considered as volume-related data.

[0060] It should be noted that in this embodiment, an interpolation integration algorithm can be used to restore the slice area of ​​the binary segmentation mask of the grain to a three-dimensional volume, and the system automatically processes the slice spacing.

[0061] S105. Visualize and output the grayscale image sequence, grain binary segmentation mask sequence, embryo binary segmentation mask sequence, and grain quantity and / or volume related data.

[0062] Specifically, this embodiment can visualize and output the generated grayscale image sequence, grain binary segmentation mask sequence, embryo binary segmentation mask sequence, and grain quantity and volume related data.

[0063] This embodiment aims to address the problems existing in related maize phenotypic analysis technologies, such as seed damage, low processing efficiency, inability to automate batch processing, insufficient computational efficiency, and difficulty in operation for non-computer professionals. To this end, an end-to-end desktop-level visualization analysis system is designed and used. This system can efficiently and accurately automate the collection, processing, and analysis of maize phenotypic data, significantly improving analysis efficiency and simplifying the operation process, making it easy for non-professionals to use.

[0064] This embodiment proposes a high-throughput visualization analysis method for maize kernel and embryo volume based on deep learning. It can obtain at least one folder based on a user-defined file input path. Each folder contains a sequence of grayscale images obtained from CT slices of maize kernels along a horizontal direction. For any folder, based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, the grayscale image sequences within the folder are preprocessed and segmented to obtain kernel binary segmentation mask sequences and embryo binary segmentation mask sequences. The number of kernels is calculated based on a created spatiotemporal consistency filtering algorithm and the kernel binary segmentation mask sequences. Furthermore, based on the kernel number, interpolation integration algorithm, kernel binary segmentation mask sequences, and embryo binary segmentation mask sequences, volume-related data for kernels and embryos are determined. Finally, the grayscale image sequences, kernel binary segmentation mask sequences, embryo binary segmentation mask sequences, kernel number, and / or volume-related data are visualized and output. This embodiment can process high-throughput data of corn kernel CT slice sequences and perform non-destructive calculation of kernel and embryo volume-related data based on computer vision technology, parallel pipeline processing mode, deep learning segmentation model, and created spatiotemporal consistency filtering algorithm. It can achieve image segmentation, visualization of kernel and embryo volume-related data, and effectively improve calculation efficiency.

[0065] based on Figure 1 This embodiment proposes a second method for high-throughput visualization analysis of maize kernel and germ volume based on deep learning. Step S105 includes: Based on the grayscale image sequence, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence, the corresponding three-dimensional models are generated and output respectively. Based on the data related to the number and volume of grains, generate statistical charts that record the data related to the number and volume of grains, and output the statistical charts.

[0066] like Figure 7 As shown, in this embodiment, the total volume of seeds, total volume of embryos, number of seeds, average volume of seeds, average volume of embryos, and volume ratio corresponding to each folder can be stored in a row record of a table, and the table can be stored in the corresponding target storage space according to the file output path set by the user. Figure 7In this context, dir_name is the directory name, embryos_volume is the total volume of the embryo, grains_volume is the total volume of the grains, grains_count is the number of grains, avg_embryos_volume is the average volume of the embryo, avg_grains_volume is the average volume of the grains, and embryos_to_grains_ratio represents the volume ratio.

[0067] like Figure 6 As shown, this embodiment can store the table in the target storage space in the format of comma-separated values ​​(CSV).

[0068] Optionally, after step S101, the method further includes: Read the dots per inch (DPI) from the image metadata of the grayscale image sequence in the folder, and convert the DPI to the corresponding physical pixel size; Provide users with the option to manually overwrite image size, receive the pixel side length and slice spacing input by the user when the image DPI is missing or inconsistent, and use the pixel side length and slice spacing input by the user to perform volume calculation; Create a global log to record the processing progress of each folder, abnormal DPI settings per inch, and / or file corruption information. When processing a folder fails, the system logs the failure information in the global log, interrupts the processing of that folder, and continues processing the next folder.

[0069] like Figure 6 As shown, this embodiment can create a global log file in .txt format and store it in the target storage space. The global log can record, for example... Figure 8 The relevant content in the chart shown.

[0070] The high-throughput visualization analysis method for maize kernel and germ volume based on deep learning proposed in this embodiment has the following technical effects: 1. Extremely high ease of use: Complex command-line operations are encapsulated into a GUI, so breeders do not need to write code. They can generate volume reports from scanned images by simply clicking the mouse, and the physical unit conversion is completed automatically.

[0071] 2. Significantly improved processing efficiency: Adopting a multi-threaded asynchronous pipeline design and extensively optimizing and adapting algorithms, compared to traditional serial scripts, under the same hardware and model configuration, the processing efficiency is improved by approximately 1124%, and the utilization of video memory is more saturated.

[0072] 3. High throughput and stability: Supports one-click processing of parent directories containing hundreds of sample folders, has a robust error capture mechanism, and is suitable for large-scale population phenotypic analysis.

[0073] 4. Accurate volume estimation: It combines deep learning-based fine segmentation with interpolation-based volume integration to achieve sub-pixel-level volume calculation accuracy, solving the problem of unclear identification of internal embryonic structures in traditional threshold segmentation methods.

[0074] Based on the above, this embodiment proposes a third method for high-throughput visualization analysis of corn kernel and embryo volume based on deep learning. In this method, the system can be developed based on Python language, the interface can be built using PyQt5, and deep learning inference can be performed using PyTorch.

[0075] Specifically, this embodiment can perform system initialization and model loading. Upon system startup, it can read the default configuration from configs.py. When the user clicks "Start Processing," the background worker thread first initializes two independent U-Net network instances: one for extracting "corn germ" and one for extracting "corn kernels." The model backbone network adopts a Visual Geometry Group (VGG) architecture, loading pre-trained weights and transferring them to the GPU.

[0076] Specifically, this embodiment can perform automated data parsing and preprocessing. After the user selects an input path, the system automatically determines the path structure. If it is a parent folder mode, the system traverses all subfolders. For each sample folder, the system first attempts to read the metadata information of the first image to obtain the DPI; if this fails, the system prompts the user to manually input the DPI or use the default value.

[0077] Specifically, this embodiment can execute a multi-threaded parallel processing flow. The system starts ThreadPoolExecutor to manage the thread pool, specifically including: Step S1: The reading thread continuously loads images from the disk, performs normalization processing, and packages them into the preprocessing queue according to the set batch size (e.g., 12).

[0078] Step S2: The prediction thread monitors the preprocessing queue (Preprocess Queue) and sends data to the GPU for inference as soon as it becomes available. The inference results (probability graph) are then pushed into the prediction queue (Predict Queue).

[0079] Step S3: The post-processing thread retrieves data from the Predict Queue, generates a binary mask based on the threshold, calculates the embryo and seed areas of the slice, and saves the mask as an image (if selected).

[0080] Specifically, this embodiment can also summarize and output the results. After single-sample processing, the system calls the statistics module to perform numerical integration on the area of ​​all slices and calculate the volume. Simultaneously, in the statistics module, to improve the accuracy of counting, the system first performs Z-axis consistency verification on the segmented grain binary segmentation mask sequence. The program sets a sliding window of size N (e.g., N=21) and calculates the intersection of all images in the grain binary segmentation mask sequence within the window layer by layer. This processing simulates physical three-dimensional core extraction, ensuring that only structures with a certain thickness and stable position in the Z-axis direction are counted. Subsequently, morphological erosion of the XY plane is performed on the processed sequence. Finally, the scipy.ndimage.label function is called to label the connected components of the three-dimensional matrix, thereby obtaining the accurate number of grains, and using this as the denominator to calculate the average volume of a single grain and the embryo / grain volume ratio. The final result is written to a CSV file in real time, and the double-layer progress bar on the interface (single-sample progress + total folder progress) is updated.

[0081] like Figure 9 As shown in the figure, this embodiment proposes a high-throughput visualization analysis system for maize kernel and germ volume based on deep learning. The system may include: The acquisition unit 101 is used to acquire at least one folder according to the file input path set by the user, and each folder includes a grayscale image sequence obtained by CT slices of corn kernels in the horizontal direction. The segmentation unit 102 is used to preprocess and segment the grayscale image sequence in any folder based on the parallel pipeline processing mode, the set batch size and the deep learning segmentation model, so as to obtain the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence. Counting unit 103 is used to count the number of seeds based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence; The determining unit 104 is used to determine the volume-related data of the grains and embryos based on the number of grains, the interpolation integration algorithm, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence. The output unit 105 is used to visualize and output grayscale image sequences, grain binary segmentation mask sequences, embryo binary segmentation mask sequences, and grain quantity and / or volume-related data.

[0082] It should be noted that the processing procedures of the acquisition unit 101, the segmentation unit 102, the counting unit 103, the determination unit 104, and the output unit 105, as well as their beneficial effects, can be found by referring to [the relevant documentation]. Figure 1 Steps S101 to S105 are not described in detail here.

[0083] Optionally, the counting unit 103 is also used for: Based on the spatiotemporal consistency filtering algorithm, each grain binary segmentation mask in the grain binary segmentation mask sequence is filtered to obtain the corresponding filtered mask sequence. Based on the image order in the filtered mask sequence, each mask in the filtered mask sequence is stacked to obtain a three-dimensional voxel matrix; The number of seeds is obtained by counting the seeds in a three-dimensional voxel matrix using a three-dimensional connected component labeling algorithm.

[0084] Optionally, the counting unit 103 is also used for: For any seed binary segmentation mask in the seed binary segmentation mask sequence, obtain the k masks before the seed binary segmentation mask, the seed binary segmentation mask, and the k masks after the seed binary segmentation mask from the seed binary segmentation mask sequence. Arrange each obtained mask according to the order of the masks to obtain the corresponding local mask sequence. Perform a logical AND operation on each mask in the local mask sequence to determine the overlapping region mask of the local mask sequence. Replace the seed binary segmentation mask in the seed binary segmentation mask sequence with the overlapping region mask to achieve filtering of the seed binary segmentation mask. The mask sequence obtained after filtering each grain binary segmentation mask is determined as the filtered mask sequence.

[0085] Optionally, the volume-related data includes total grain volume, total embryo volume, average grain volume, average embryo volume, and volume ratio, where the volume ratio is the ratio of average grain volume to average embryo volume.

[0086] Optionally, the determining unit 104 is also used for: Based on the interpolation integration algorithm, the volume of each grain binary segmentation mask in the grain binary segmentation mask sequence and the volume of each embryo binary segmentation mask in the embryo binary segmentation mask sequence are calculated to obtain the total grain volume and the total embryo volume. Divide the total volume of grains and the total volume of embryos by the number of grains to obtain the average volume of grains and the average volume of embryos, respectively. Calculate the volume ratio based on the average volume of the grain and the average volume of the embryo; The total volume of grains, the total volume of embryos, the average volume of grains, the average volume of embryos, and the volume ratio are all considered as volume-related data.

[0087] Optionally, the segmentation unit 102 is also used for: Based on a parallel pipeline processing mode and a set batch size, a deep learning segmentation model is invoked to preprocess the grayscale image sequences in the folder, perform dual-channel parallel segmentation of seeds and embryos, and post-processing to obtain the corresponding seed binary segmentation mask sequence, embryo binary segmentation mask sequence, and color segmentation mask sequence. Figure 2 Value segmentation mask sequence; The above system also includes: The storage unit is used to store the grain binary segmentation mask sequence, embryo binary segmentation mask sequence, and color image according to the file output path set by the user. Figure 2 The value segmentation mask sequence is saved to the corresponding target storage space.

[0088] Optionally, output unit 105 is also used for: Based on the grayscale image sequence, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence, the corresponding three-dimensional models are generated and output respectively. Based on the data related to the number and volume of grains, generate statistical charts that record the data related to the number and volume of grains, and output the statistical charts.

[0089] Optionally, the above system also includes: The conversion unit is used to read the dots per inch (DPI) from the image metadata of the grayscale image sequence of the folder after obtaining at least one folder according to the file input path set by the user, and convert the DPI into the corresponding physical pixel size. The receiving unit is used to provide users with the option to manually overwrite the image size, receive the pixel side length and slice spacing input by the user when the image DPI is missing or inconsistent, and use the pixel side length and slice spacing input by the user to perform volume calculation. Create a unit to create a global log. The global log is used to record the processing progress of each folder, abnormal DPI settings per inch, and / or file corruption information. The interrupt unit is used to record failure-related information in the global log when folder processing fails, interrupt folder processing, and continue processing the next folder.

[0090] Optionally, the above system also includes: The error reporting unit is used to determine invalid input after obtaining at least one folder based on the file input path set by the user. If any folder contains both images and subfolders, or contains neither images nor subfolders, then the input is considered invalid. When the folder is determined to be invalid input or other preset errors occur, the unit records the failure information in the created global log, reports the error through the interface, and at least terminates the processing of the folder.

[0091] The high-throughput visualization analysis system for maize kernel and embryo volume based on deep learning proposed in this embodiment can obtain at least one folder according to the file input path set by the user. Each folder contains a sequence of grayscale images obtained by computed tomography (CT) slices of maize kernels along the horizontal direction. For any folder, based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, the grayscale image sequences in the folder are preprocessed and segmented to obtain a kernel binary segmentation mask sequence and an embryo binary segmentation mask sequence. The number of kernels is counted based on the created spatiotemporal consistency filtering algorithm and the kernel binary segmentation mask sequence. And, based on the number of kernels, the interpolation integration algorithm, the kernel binary segmentation mask sequence, and the embryo binary segmentation mask sequence, the volume-related data of kernels and embryos are determined. The grayscale image sequence, the kernel binary segmentation mask sequence, the embryo binary segmentation mask sequence, the number of kernels and / or the volume-related data are visualized and output. This embodiment can process high-throughput data of corn kernel CT slice sequences and perform non-destructive calculation of kernel and embryo volume-related data based on computer vision technology, parallel pipeline processing mode, deep learning segmentation model, and created spatiotemporal consistency filtering algorithm. It can achieve image segmentation, visualization of kernel and embryo volume-related data, and effectively improve calculation efficiency.

[0092] The deep learning-based high-throughput visualization analysis system for corn kernel and germ volume in this embodiment is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0093] This invention also provides a computer device having the above-described features. Figure 9 The image shows a high-throughput visualization analysis system for maize kernel and germ volume based on deep learning.

[0094] Please see Figure 10The present invention provides a schematic diagram of the structure of a computer device according to an optional embodiment. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 10 Take a processor 10 as an example.

[0095] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0096] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0097] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] Memory 20 may include volatile memory, such as random access memory. Memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. Memory 20 may also include combinations of the above types of memory.

[0099] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0100] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-throughput visualization analysis method for maize kernel and germ volume based on deep learning, characterized in that, include: At least one folder is obtained according to the file input path set by the user, and each folder includes a grayscale image sequence obtained by CT slices of corn kernels in the horizontal direction; For any of the folders, based on the parallel pipeline processing mode, the set batch size and the deep learning segmentation model, the grayscale image sequence in the folder is preprocessed and segmented to obtain the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence. The number of seeds is counted based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence; the volume-related data of seeds and embryos are determined based on the number of seeds, the interpolation integration algorithm, the seed binary segmentation mask sequence and the embryo binary segmentation mask sequence. The grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data are visualized and output.

2. The method according to claim 1, characterized in that, The method for calculating the number of seeds based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence includes: Based on the spatiotemporal consistency filtering algorithm, each seed binary segmentation mask in the seed binary segmentation mask sequence is filtered to obtain the corresponding filtered mask sequence. Based on the image order in the filtered mask sequence, each mask in the filtered mask sequence is stacked to obtain a three-dimensional voxel matrix; The number of seeds is obtained by counting the seeds in the three-dimensional voxel matrix using a three-dimensional connected component labeling algorithm.

3. The method according to claim 2, characterized in that, The spatiotemporal consistency filtering algorithm is used to filter each seed binary segmentation mask in the seed binary segmentation mask sequence to obtain a corresponding filtered mask sequence, including: For any seed binary segmentation mask in the seed binary segmentation mask sequence, obtain k masks before the seed binary segmentation mask, the seed binary segmentation mask, and k masks after the seed binary segmentation mask from the seed binary segmentation mask sequence. Arrange each obtained mask according to the order of mask sequence to obtain a corresponding local mask sequence. Perform a logical AND operation on each mask in the local mask sequence to determine the overlapping region mask of the local mask sequence. Replace the seed binary segmentation mask in the seed binary segmentation mask sequence with the overlapping region mask to achieve filtering of the seed binary segmentation mask. The mask sequence obtained by filtering each of the binary segmentation masks of the seeds is determined as the filtered mask sequence.

4. The method according to claim 1, characterized in that, The volume-related data includes total grain volume, total embryo volume, average grain volume, average embryo volume, and volume ratio, where the volume ratio is the ratio of the average grain volume to the average embryo volume.

5. The method according to claim 4, characterized in that, The step of determining the volume-related data of seeds and embryos based on the seed quantity, interpolation integration algorithm, seed binary segmentation mask sequence, and embryo binary segmentation mask sequence includes: According to the interpolation integration algorithm, the volume of each grain binary segmentation mask in the grain binary segmentation mask sequence and the volume of each embryo binary segmentation mask in the embryo binary segmentation mask sequence are calculated to obtain the total grain volume and the total embryo volume. The average grain volume and the average embryo volume are obtained by dividing the total grain volume and the total embryo volume by the number of grains, respectively. The volume ratio is calculated based on the average volume of the grain and the average volume of the embryo. The total volume of the grains, the total volume of the embryos, the average volume of the grains, the average volume of the embryos, and the volume ratio are collectively used as the volume-related data.

6. The method according to claim 1, characterized in that, The method, based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, preprocesses and segments the grayscale image sequence in the folder to obtain a seed binary segmentation mask sequence and an embryo binary segmentation mask sequence, including: Based on the parallel pipeline processing mode and the set batch size, the deep learning segmentation model is called to preprocess, perform dual-channel parallel segmentation of seeds and embryos in the grayscale image sequence in the folder, and perform post-processing to obtain the corresponding seed binary segmentation mask sequence, embryo binary segmentation mask sequence and color image binary segmentation mask sequence. The method further includes: According to the file output path set by the user, the seed binary segmentation mask sequence, the embryo binary segmentation mask sequence, and the color image binary segmentation mask sequence are saved to the corresponding target storage space.

7. The method according to claim 1, characterized in that, The visualization output of the grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data includes: Based on the grayscale image sequence, the grain binary segmentation mask sequence, and the embryo binary segmentation mask sequence, respectively, generate and output the corresponding three-dimensional models; Based on the seed quantity and volume-related data, a statistical chart recording the seed quantity and volume-related data is generated and output.

8. The method according to claim 1, characterized in that, After obtaining at least one folder based on the file input path set by the user, the method further includes: Read the dots per inch (DPI) from the image metadata of the grayscale image sequence in the folder, and convert the DPI into the corresponding physical pixel size; Provide the user with a manual overlay image size option, receive the pixel side length and slice spacing input by the user when the image DPI is missing or inconsistent, and use the pixel side length and slice spacing input by the user to perform volume calculation; Create a global log, which is used to record the processing progress of each folder, abnormal DPI settings, and / or file corruption information; When processing of the folder fails, the global log records the failure information, interrupts the processing of the folder, and continues to process the next folder.

9. The method according to any one of claims 1 to 8, characterized in that, After obtaining at least one folder based on the file input path set by the user, the method further includes: For any of the folders, if the folder contains both images and subfolders, or if the folder contains neither images nor subfolders, then it is determined to be invalid input; When the folder is determined to be invalid input or other preset errors occur, the created global log records the failure information, reports the error through the interface, and at least terminates the processing of the folder.

10. A high-throughput visualization analysis system for maize kernel and germ volume based on deep learning, characterized in that, include: The acquisition unit is used to acquire at least one folder according to the file input path set by the user, and each folder includes a grayscale image sequence obtained by performing a computed tomography CT slice of corn kernels along the horizontal direction. The segmentation unit is used to preprocess and segment the grayscale image sequence in any folder based on a parallel pipeline processing mode, a set batch size, and a deep learning segmentation model, so as to obtain a seed binary segmentation mask sequence and an embryo binary segmentation mask sequence. A counting unit is used to count the number of seeds based on the created spatiotemporal consistency filtering algorithm and the seed binary segmentation mask sequence. The determining unit is used to determine the volume-related data of the seeds and embryos based on the seed quantity, the interpolation integration algorithm, the seed binary segmentation mask sequence, and the embryo binary segmentation mask sequence. The output unit is used to visualize and output the grayscale image sequence, the grain binary segmentation mask sequence, the embryo binary segmentation mask sequence, the grain quantity and / or the volume-related data.

Images