Spatial rice leaf tip detection and growth trajectory generation method, system and storage medium

Abstract

The present invention discloses a method, system and storage medium for detecting and generating a growth trajectory of rice leaf tips in space, including: based on a plurality of target training images containing rice in space, training a preset target detection model for detecting rice leaf tips in space to obtain a trained target detection model; inputting images of rice to be detected in space at different times into the trained target detection model to obtain a target detection frame where the leaf tip of rice to be detected in each image to be detected is located; according to each target detection frame corresponding to any leaf tip of rice to be detected, determining the spatial coordinates of the leaf tip in a preset coordinate system, and generating the growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip. The present invention can accurately locate the leaf tip of rice in space and can concisely and efficiently realize the generation of the leaf tip growth trajectory, providing technical support for in-depth research on the growth movement rhythm of rice plants.

Description

Background Art

[0002] With the rapid development of my country's manned space technology and life science theory and technology, a series of emerging disciplines generated and developed by manned space exploration, represented by space life science, have gradually developed, making important contributions to the development of my country's space industry. As the Chinese space station continues to operate and the operating platform continues to improve, it provides a large amount of meaningful experimental data for the ground, especially the images and video data of plant and animal growth captured by the camera, which provides scientists with effective real data support in space environments such as weightlessness, radiation and magnetic fields for in-depth research on space biological rhythms and higher plants.

[0003] Among them, research on higher plants, such as recording and analyzing the entire growth and development process of rice germination, flowering, pollination, and fruiting under different photoperiod conditions in weightless space, can provide new evidence for analyzing and clarifying the role of gravity in photoperiod-induced flowering of plants and the impact of gravity on the rhythm of plant growth cycles. In particular, research on the generation of rice leaf tip swing growth trajectories, quantitative statistics of plant operation and growth characteristics, deepening the quantitative understanding of biological and botanical issues, and exploring the relationship between external stress and strain in the growth process of plants under specific environments are of great significance for studying the characteristics of plant operation cycles and growth rhythms.

[0004] However, the industry has not yet developed a research program for the generation of rice leaf tip trajectories. Existing programs mainly rely on manual work to complete the labeling of rice leaf tips frame by frame. However, manual labeling is time-consuming, inefficient, and costly. The amount of labeled data is large and the time required is long. Even if the position information of the rice leaf tip at each moment is obtained, it is difficult to intuitively analyze and display the growth trajectory of rice to statistically analyze the periodic growth rhythm.

[0005] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a method, system and storage medium for spatial rice leaf tip detection and growth trajectory generation.

[0007] The technical scheme of the spatial rice leaf tip detection and growth trajectory generation method of the present invention is as follows:

[0008] S1. Based on a plurality of target training images containing space rice, a preset target detection model for detecting the tip of space rice leaf is trained to obtain a trained target detection model;

[0009] S2, inputting the images of rice in the space to be detected at different times into the trained target detection model, and obtaining the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located;

[0010] S3. According to each target detection frame corresponding to any leaf tip of the rice to be detected, determine the spatial coordinates of the leaf tip in a preset coordinate system, and generate a growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip.

[0011] The beneficial effects of the spatial rice leaf tip detection and growth trajectory generation method of the present invention are as follows:

[0012] The method of the present invention can accurately locate the leaf tip of rice in space and can generate the growth trajectory of the leaf tip in a simple and efficient manner, thereby providing data services for the research on the mechanism of space science experiments and providing technical support for scientific researchers to conduct in-depth research on the growth and movement rhythm of rice plants.

[0013] Based on the above scheme, the spatial rice leaf tip detection and growth trajectory generation method of the present invention can also be improved as follows.

[0014] Furthermore, it also includes:

[0015] A plurality of original training images containing space rice are obtained, and data enhancement processing is performed on each of the original training images to obtain a plurality of target training images.

[0016] Furthermore, it also includes:

[0017] Each leaf tip in each target training image is labeled respectively to obtain a plurality of training annotation boxes containing the leaf tips corresponding to each target training image.

[0018] Further, step S1 comprises:

[0019] S11, inputting any target training image into the preset target detection model, obtaining each training detection frame containing a leaf tip in the any target training image, and obtaining a loss value of the any target training image according to a training annotation frame and a training detection frame of each identical leaf tip in the any target training image, until a loss value of each target training image is obtained;

[0020] S12. Based on all loss values, the network parameters of the preset target detection model are optimized to obtain an optimized target detection model. The optimized target detection model is used as the preset target detection model and the process returns to step S11 until the optimized target detection model meets the preset training conditions, and the optimized target detection model is determined as the trained target detection model.

[0021] Furthermore, the preset target detection model includes: sequentially connecting a backbone network of fused position attention, a multi-scale feature pyramid network, and a candidate region generation network; inputting any target training image into the preset target detection model, and obtaining the step of each training detection frame containing a leaf tip of any target training image, including:

[0022] Inputting any target training image into the backbone network of the fused position attention to obtain four first feature maps of different resolutions of any target training image;

[0023] Inputting four first feature maps of different resolutions of any target training image into a multi-scale feature pyramid network to obtain five second feature maps of different resolutions of any target training image;

[0024] The five second feature maps with different resolutions of any target training image are input into a candidate region generation network to obtain each training detection box containing a leaf tip of any target training image.

[0025] Further, any image to be tested is a red light background image to be tested or multiple white light background images to be tested; according to any target detection frame of the rice in the space to be tested, the step of determining the spatial coordinates of the leaf tip in the target detection frame in the preset coordinate system comprises:

[0026] When any of the target detection frames is a target detection frame corresponding to a white light background image to be tested, based on color information, a rice plant body is extracted from the white light background image to be tested corresponding to the any of the target detection frames to obtain a plant extraction image corresponding to the white light background image to be tested, and the any of the target detection frames is mapped to the plant extraction image corresponding to the white light background image to be tested to obtain a target detection light mapping area of ​​the any of the target detection frames, and a Harris-Laplace corner point detector is used to extract all corner point features in the target detection light mapping area of ​​the any of the target detection frames, and a KMeans clustering algorithm is used to cluster all corner point features in the target detection light mapping area of ​​the any of the target detection frames to obtain the spatial coordinates of the leaf tip in the any of the target detection frames in the preset coordinate system;

[0027] When any of the target detection frames is a target detection frame corresponding to the image to be detected with a red light background, the center point position of any of the target detection frames is determined as the spatial coordinate of the blade tip in the preset coordinate system.

[0028] Furthermore, it also includes:

[0029] Based on the nearest neighbor matching method of the spatial position, each target detection frame corresponding to any leaf tip is determined.

[0030] The technical solution of the spatial rice leaf tip detection and growth trajectory generation system of the present invention is as follows:

[0031] Includes: training module, detection module and generation module;

[0032] The training module is used to: train a preset target detection model for detecting the tip of rice leaves in space based on a plurality of target training images containing rice in space, to obtain a trained target detection model;

[0033] The detection module is used to: input the images of rice in the space to be detected at different times into the trained target detection model to obtain the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located;

[0034] The generation module is used to: determine the spatial coordinates of any leaf tip of the rice to be detected in the preset coordinate system according to each target detection frame corresponding to the leaf tip, and generate the growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip.

[0035] The beneficial effects of the spatial rice leaf tip detection and growth trajectory generation system of the present invention are as follows:

[0036] The system of the present invention can accurately locate the leaf tip of rice in space and can generate the growth trajectory of the leaf tip in a simple and efficient manner, thereby providing data services for the research on the mechanism of space science experiments and providing technical support for scientific researchers to conduct in-depth research on the growth and movement rhythm of rice plants.

[0037] Based on the above solution, the spatial rice leaf tip detection and growth trajectory generation system of the present invention can also be improved as follows.

[0038] Further, it also includes: a processing module;

[0039] The processing module is used to obtain a plurality of original training images containing space rice, and perform data enhancement processing on each original training image to obtain a plurality of target training images.

[0040] A technical solution of a storage medium of the present invention is as follows:

[0041] The storage medium stores instructions, and when a computer reads the instructions, the computer executes the steps of the spatial rice leaf tip detection and growth trajectory generation method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram showing a flow chart of an embodiment of a method for detecting and generating growth trajectories of rice leaves in space provided by the present invention;

[0043] Figure 2A schematic diagram of leaf tip annotation in an embodiment of the method for spatial rice leaf tip detection and growth trajectory generation provided by the present invention is shown;

[0044] Figure 3 A schematic diagram showing the structure of a preset target detection model in an embodiment of the method for detecting and generating growth trajectories of spatial rice leaf tips provided by the present invention is shown;

[0045] Figure 4 A schematic diagram showing the structure of the original backbone network in an embodiment of the method for detecting and generating growth trajectories of spatial rice leaf tips provided by the present invention is shown;

[0046] Figure 5 A schematic diagram showing the structure of a backbone network integrating position attention in an embodiment of a spatial rice leaf tip detection and growth trajectory generation method provided by the present invention;

[0047] Figure 6 A schematic diagram showing a nearest neighbor matching method in an embodiment of the spatial rice leaf tip detection and growth trajectory generation method provided by the present invention;

[0048] Figure 7 A schematic diagram showing the detection result of the space rice to be detected in the embodiment of the space rice leaf tip detection and growth trajectory generation method provided by the present invention is shown;

[0049] Figure 8 A schematic diagram showing changes in loss and accuracy during the training process of a target detection model in an embodiment of the spatial rice leaf tip detection and growth trajectory generation method provided by the present invention is shown;

[0050] Fig. 9 The heat map before and after the target detection model training in the embodiment of the spatial rice leaf tip detection and growth trajectory generation method provided by the present invention is shown;

[0051] Fig.10 A schematic diagram of leaf tip positioning errors in the horizontal and vertical directions in an embodiment of the spatial rice leaf tip detection and growth trajectory generation method provided by the present invention is shown;

[0052] Fig.11 A structural schematic diagram of an embodiment of a spatial rice leaf tip detection and growth trajectory generation system provided by the present invention is shown. DETAILED DESCRIPTION

[0053] Figure 1 FIG. 2 is a flow chart showing an embodiment of the method for detecting rice leaf tips and generating growth trajectories in space provided by the present invention. Figure 1 As shown, the following steps are included:

[0054] S1. Based on a plurality of target training images containing space rice, a preset target detection model for detecting the tip of space rice leaf is trained to obtain a trained target detection model.

[0055] Among them, ① the target training image is: an image containing space rice, which is used to train the target detection model. ② the preset target detection model is: a pre-built model for detecting the tip of space rice leaves, and the detection target type of the model is: the tip of space rice leaves; the input of the model is the target training image, and the output is the target detection frame. ③ the trained target detection model is: a trained model for detecting the tip of space rice leaves.

[0056] S2. Input the images of rice in the space to be detected at different times into the trained target detection model to obtain the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located.

[0057] Among them, ① the spatial rice to be detected is: the spatial rice that needs to perform leaf tip detection and leaf tip growth trajectory generation in this embodiment. ② The number of images to be tested is multiple, which are images of the spatial rice to be detected at different times. ③ The target detection frame is: the target frame containing the leaf tip of the spatial rice to be detected; each image to be tested corresponds to at least one target detection frame.

[0058] It should be noted that, since the growth changes of rice are slow and weak, in order to better analyze the growth trajectory of the leaf tip, sampling is performed every 50 frames in the video data of the rice in the space to be detected to obtain the test images of the rice in the space to be detected at different times. It can also be selected according to actual needs, and there is no restriction here.

[0059] S3. According to each target detection frame corresponding to any leaf tip of the rice to be detected, determine the spatial coordinates of the leaf tip in a preset coordinate system, and generate a growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip.

[0060] Among them, ① the rice in the space to be detected includes multiple leaf tips, each leaf tip corresponds to a growth trajectory. ② With the upper left corner of the image as the coordinate origin, the first coordinate axis points to the right, and the second coordinate axis points downward, a preset coordinate system is constructed.

[0061] Preferably, it also includes:

[0062] A plurality of original training images containing space rice are obtained, and data enhancement processing is performed on each of the original training images to obtain a plurality of target training images.

[0063] Among them, ① the original training image is: an image containing space rice captured from the space rice video, with a default size of 1920×1080. ② Data enhancement processing includes but is not limited to: random cropping of the image to 512×512, random translation, flipping, brightness change, etc.

[0064] Preferably, it also includes:

[0065] Each leaf tip in each target training image is labeled respectively to obtain a plurality of training annotation boxes containing the leaf tips corresponding to each target training image.

[0066] The annotation method is to draw a circle with the leaf tip in the image as the center, and then take the circumscribed square of the circle to ensure that the leaf tip is always in the center of the target frame, which can help the network learn this latent spatial position feature and make the predicted leaf tip as close to the center of the frame as possible. Specifically, Figure 2 As shown, the circular annotation radius is R, and the leaf tip (x, y) is always kept in the center area of ​​the annotation box, which also helps the network to learn this implicit prior information during the learning process.

[0067] Preferably, step S1 includes:

[0068] S11. Input any target training image into the preset target detection model to obtain each training detection frame containing a leaf tip in the any target training image, and obtain the loss value of the any target training image according to the training annotation frame and the training detection frame of each identical leaf tip in the any target training image, until the loss value of each target training image is obtained.

[0069] Specifically, any target training image is input into a preset target detection model to obtain each training detection frame containing a leaf tip in the target training image, and based on the training annotation frame and the training detection frame of any leaf tip in the target training image, a training loss value of the said any leaf tip in the target training image is obtained. The above process is repeated to obtain and based on the training loss value of each leaf tip in the target training image, a loss value of the target training image is obtained until the loss value of each target training image is obtained.

[0070] S12. Based on all loss values, the network parameters of the preset target detection model are optimized to obtain an optimized target detection model. The optimized target detection model is used as the preset target detection model and the process returns to step S11 until the optimized target detection model meets the preset training conditions, and the optimized target detection model is determined as the trained target detection model.

[0071] Among them, the preset training condition is: reaching the maximum number of iterative training times (which can be set according to actual needs and is not limited here).

[0072] Specifically, all loss values ​​are used to optimize the network parameters of the preset target detection model to obtain an optimized target detection model. The optimized target detection model is used as the preset target detection model and returns to repeatedly execute step S11 until the optimized target detection model reaches the maximum number of iterative training times, and the optimized target detection model is determined as the trained target detection model.

[0073] It should be noted that the iterative training process of the target detection model and the specific implementation method of optimizing the network parameters by the loss value are existing technologies and will not be elaborated here.

[0074] Preferably, the preset target detection model includes: a backbone network that integrates position attention, a multi-scale feature pyramid network, and a candidate region generation network that are sequentially connected.

[0075] Among them, the overall structure diagram of the preset target detection model is as follows Figure 3 As shown. Specifically, in order to fully learn the generalization features of different backgrounds and leaf tip instances, in this embodiment, the preset target detection model uses the classic ResNet50 network as the basic backbone network, adds position attention to the basic backbone network, and loads the parameters pre-trained on the large public data set ImageNet as the initial values ​​of the network training. The output feature maps with the number of channels of 256, 512, 1024, and 2048 are connected to the multi-scale feature pyramid network, so that the network learns the leaf tip area features of different scales. The feature map with the number of channels outputted is then input into the candidate region generation network to generate the initial candidate box of the leaf tip. The initial anchor ratio is set to 1:2, 1:1, and 2:1. The candidate box initially learned by the region generation network is then input into the three-level cascade RCNN network for refined learning and correction. Each RCNN detection head contains two convolution branches, which respectively complete classification and positioning. As the quality of the candidate boxes gradually improves, the IOU thresholds are set to 0.5, 0.6, and 0.7 at the three levels, respectively, to achieve refined learning of target box positioning and obtain a training detection box containing each leaf tip in the image.

[0076] The step of inputting any target training image into the preset target detection model to obtain each training detection frame containing a leaf tip of the any target training image comprises:

[0077] Input any target training image into the backbone network of the fused position attention to obtain four first feature maps of different resolutions of any target training image.

[0078] Among them, the backbone network integrating position attention is: integrating position attention into the original backbone network. Figure 4 As shown in Figure 1, the original backbone network Resnet50 consists of basic structures such as BasicBlock and BottleNeck. Figure 5 As shown in the figure, the backbone network of the fusion position attention is improved in the BottleNeck structure, and the position attention is integrated, but the number of input and output channels remains unchanged. The input of the backbone network of the fusion position attention is: the target training image (image of size 512×512); the output of the backbone network of the fusion position attention is: the first feature map of four different resolutions.

[0079] It should be noted that the sizes of the first feature maps of the four different resolutions are: [B, 128, 128, 256], [B, 64, 64, 512], [B, 32, 32, 1024], [B, 16, 16, 2048], respectively; where B is the batch size set in the experiment, and B=4 is set by default in this embodiment.

[0080] The four first feature maps of different resolutions of any target training image are input into a multi-scale feature pyramid network to obtain five second feature maps of different resolutions of any target training image.

[0081] Among them, the input of the multi-scale feature pyramid network is: four first feature maps of different resolutions output by the backbone network that integrates position attention; the output of the multi-scale feature pyramid network is: five second feature maps of different resolutions.

[0082] It should be noted that after upsampling, fusion and convolution processing of adjacent first feature maps, five second feature maps with different resolutions are obtained, and their sizes are: [4, 128, 128, 256], [4, 64, 64, 256], [4, 32, 32, 256], [4, 16, 16, 256], [4, 16, 16, 256].

[0083] The five second feature maps with different resolutions of any target training image are input into a candidate region generation network to obtain each training detection box containing a leaf tip of any target training image.

[0084] The input of the candidate region generation network is: five second feature maps with different resolutions; the output of the candidate region generation network is: each training detection box containing the leaf tip in the target training image.

[0085] It should be noted that five second feature maps with different resolutions are input into the candidate region generation network, anchor pre-selection boxes of different scales are generated on the input second feature maps with different resolutions, the feature regions corresponding to the anchors are classified and regressed, the precise position information of the target is learned, and each training detection box containing the leaf tip in the target training image is obtained.

[0086] Preferably, any image to be tested is an image to be tested with a red light background or multiple images to be tested with a white light background.

[0087] The step of determining the spatial coordinates of the leaf tip in the target detection frame in the preset coordinate system according to any target detection frame of the rice in the space to be detected comprises:

[0088] When any of the target detection frames is a target detection frame corresponding to an image to be tested with a white light background, based on color information, rice plant extraction is performed on the image to be tested with a white light background corresponding to the any of the target detection frames to obtain a plant extraction image corresponding to the image to be tested with a white light background, and the any of the target detection frames is mapped to the plant extraction image corresponding to the image to be tested with a white light background to obtain a target detection light mapping area of ​​the any of the target detection frames, and a Harris-Laplace corner detector is used to extract all corner point features in the target detection light mapping area of ​​the any of the target detection frames, and a KMeans clustering algorithm is used to cluster all corner point features in the target detection light mapping area of ​​the any of the target detection frames to obtain the spatial coordinates of the leaf tip in the any of the target detection frames in the preset coordinate system.

[0089] Specifically, when any target detection frame is a target detection frame corresponding to a white light background image to be tested, ① based on color information, rice plant body extraction is performed on the white light background image to be tested corresponding to any target detection frame to obtain a plant extraction image corresponding to the white light background image to be tested; ② any target detection frame is mapped to the plant extraction image corresponding to the white light background image to be tested to obtain a target detection light mapping area of ​​any target detection frame; ③ using a Harris-Laplace corner point detector, all corner point features in the target detection light mapping area of ​​any target detection frame are extracted; and ④ using a KMeans clustering algorithm, all corner point features in the target detection light mapping area of ​​any target detection frame are clustered to obtain the spatial coordinates of the leaf tip in any target detection frame in the preset coordinate system.

[0090] When any of the target detection frames is a target detection frame corresponding to the image to be detected with a red light background, the center point position of any of the target detection frames is determined as the spatial coordinate of the blade tip in the preset coordinate system.

[0091] Preferably, it also includes:

[0092] Based on the nearest neighbor matching method of the spatial position, each target detection frame corresponding to any leaf tip is determined.

[0093] Specifically, the spatial coordinates of the leaf tips in all target detection frames are obtained, and from all the spatial coordinates, the spatial coordinates corresponding to any leaf tip are determined by utilizing the nearest neighbor matching method of the spatial position (since the spatial coordinates and the target detection frame are in a one-to-one correspondence, each target detection frame corresponding to any leaf tip is obtained).

[0094] It should be noted that, ① due to the high frame rate of spatial rice video data, the spatial position of the leaf tip between two adjacent frames (two adjacent images to be tested) changes slightly, and there is a certain distance interval between different leaf tips. Therefore, the nearest neighbor matching method using spatial position is adopted to obtain the optimal matching relationship with the minimum distance difference. Finally, the growth trajectory of each leaf tip in multiple consecutive frames is obtained. Based on the generated trajectory data, biological laws such as periodic growth swings of rice plants can be analyzed to provide technical support for further biological research. ② Figure 6 FIG. 1 shows a schematic diagram of the nearest neighbor matching method. For the blade tip target 1, the matching blade tip target ID in the adjacent frame is calculated. The specific calculation is: Where x1 is the spatial coordinate of the target blade tip to be matched in the current frame, x i ′ is the spatial coordinate of the i-th leaf tip in the frame before the current frame;

[0095] Since the growth process of rice leaf tips is slow, the movement changes of each leaf tip between adjacent frames in continuously shot video segments are extremely small. Therefore, the algorithm can achieve concise and efficient post-matching and growth trajectory generation, avoiding the parameter learning optimization process in the deep learning network.

[0096] In addition, in this embodiment, the following points need to be explained:

[0097] ① Algorithm parameter setting. The training of the target detection model uses the SGD optimizer, the batch size is set to 4, the learning rate is set to 0.0001, and the number of training rounds is set to 20 rounds. The trained target detection model completes reasoning on all the data (including labeled and unlabeled) obtained by extracting frames. According to the reasoning area score, the iou thresholds of the non-maximum suppression operation of the region generation network and the RCNN network are adaptively adjusted to 0.7 and 0.1 respectively, effectively removing redundant results. Thus, the scheme of this embodiment completes the auxiliary output of the tip information of the leaf at all times, which can effectively reduce labor costs. In terms of the division of the data set, the video segments of several adjacent times are extracted, and a total of 1127 images are obtained. 965 images are selected as training images, including 800 white light background images and 165 red light background images. The images to be tested of the rice in the space to be detected include 162 red light data, and the tip of the leaf has two obvious growth trajectory changes. In addition, there are two different time periods, and two video segments with large differences in background illumination and other conditions are used for model generalization verification analysis.

[0098] ②Verification and analysis of experimental results.

[0099] 1) Figure 7 The image detection results after two typical trajectory changes in the image of rice to be detected are shown. Figure 7 It can be seen that for leaf tips in different forms, target detection can be achieved stably and accurately, and the central area of ​​the target detection frame basically coincides with the leaf tip.

[0100] 2) Changes in loss and accuracy during the training of the target detection model Figure 8 After the final model converges, the detection accuracy mAP is increased from about 75% to 83.7%.

[0101] 3) Fig. 9 The heat map of the target detection model before and after training is shown. Visual analysis of the heat map before and after training shows that the interference of cluttered background is effectively suppressed during the model network learning process.

[0102] 4) This embodiment further analyzes the positioning error on the video segment corresponding to the rice in the space to be detected. Fig.10 The horizontal and vertical positioning errors of the leaf tip are shown in the figure respectively. The calculated horizontal positioning error is 0.21 pixels, and the vertical positioning error is 0.03 pixels. It can be seen that the positioning error is extremely small and will not affect the information that scientists focus on, such as the growth laws and biological rhythms of rice plants.

[0103] The technical solution of this embodiment can accurately locate the leaf tip of rice in space and can generate the growth trajectory of the leaf tip in a simple and efficient manner, thereby providing data services for the research on the mechanism of space science experiments and providing technical support for scientific researchers to conduct in-depth research on the growth and movement rhythm of rice plants.

[0104] Fig.11 FIG. 1 is a schematic diagram showing a structure of an embodiment of a spatial rice leaf tip detection and growth trajectory generation system provided by the present invention. Fig.11 As shown, the system 200 includes: a training module 210 , a detection module 220 and a generation module 230 .

[0105] The training module 210 is used to: train a preset target detection model for detecting the tip of rice leaves in space based on a plurality of target training images containing rice in space, to obtain a trained target detection model;

[0106] The detection module 220 is used to: input the images of rice in the space to be detected at different times into the trained target detection model to obtain the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located;

[0107] The generating module 230 is used to determine the spatial coordinates of any leaf tip of the rice to be detected in a preset coordinate system according to each target detection frame corresponding to the leaf tip, and generate a growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip.

[0108] Preferably, it also includes: a processing module;

[0109] The processing module is used to obtain a plurality of original training images containing space rice, and perform data enhancement processing on each original training image to obtain a plurality of target training images.

[0110] The technical solution of this embodiment can accurately locate the leaf tip of rice in space and can generate the growth trajectory of the leaf tip in a simple and efficient manner, thereby providing data services for the research on the mechanism of space science experiments and providing technical support for scientific researchers to conduct in-depth research on the growth and movement rhythm of rice plants.

[0111] The above-mentioned parameters and steps of each module to implement the corresponding functions in the embodiment of the spatial rice leaf tip detection and growth trajectory generation system 200 provided by the present invention can refer to the parameters and steps in the embodiment of the spatial rice leaf tip detection and growth trajectory generation method provided above, and will not be repeated here.

[0112] A storage medium provided by an embodiment of the present invention includes: instructions stored in the storage medium, and when a computer reads the instructions, the computer executes the steps of a method for detecting and generating growth trajectories of rice leaf tips in space. For details, please refer to the parameters and steps in the embodiment of the method for detecting and generating growth trajectories of rice leaf tips in space provided above, which will not be described in detail here.

[0113] Computer storage media such as USB flash drives, mobile hard disks, etc.

[0114] Those skilled in the art will appreciate that the present invention may be implemented as a method, a system, and a storage medium.

[0115] Therefore, the present invention can be specifically implemented in the following forms, that is, it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code. Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and cannot be construed as limiting the present invention, and those skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A spatial rice leaf tip detection and growth trajectory generation method, characterized in that: include: S1. Based on a plurality of target training images containing space rice, a preset target detection model for detecting the tip of space rice leaf is trained to obtain a trained target detection model; S2, inputting the images of rice in the space to be detected at different times into the trained target detection model, and obtaining the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located; S3, according to each target detection frame corresponding to any leaf tip of the rice in the space to be detected, determining the spatial coordinates of the leaf tip in a preset coordinate system, and generating a growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip; Step S1 includes: S11, inputting any target training image into the preset target detection model, obtaining each training detection frame containing a leaf tip in the any target training image, and obtaining a loss value of the any target training image according to a training annotation frame and a training detection frame of each identical leaf tip in the any target training image, until a loss value of each target training image is obtained; S12, based on all the loss values, optimizing the network parameters of the preset target detection model to obtain an optimized target detection model, using the optimized target detection model as the preset target detection model and returning to execute step S11, until the optimized target detection model meets the preset training conditions, determining the optimized target detection model as the trained target detection model; The preset target detection model includes: a backbone network fused with position attention, a multi-scale feature pyramid network and a candidate region generation network connected in sequence; a step of inputting any target training image into the preset target detection model to obtain each training detection frame containing a leaf tip of any target training image, including: Inputting any target training image into the backbone network of the fused position attention to obtain four first feature maps of different resolutions of any target training image; Among them, the backbone network fused with position attention integrates position attention in the BottleNeck structure of the backbone network, and the number of input and output channels of the backbone network fused with position attention remains unchanged; the sizes of the first feature maps of four different resolutions are: [B, 128, 128, 256], [B, 64, 64, 512], [B, 32, 32, 1024], [B, 16, 16, 2048]; B is the batch size set in the experiment; Inputting four first feature maps of different resolutions of any target training image into a multi-scale feature pyramid network to obtain five second feature maps of different resolutions of any target training image; Among them, after upsampling, fusion and convolution processing of adjacent first feature maps, five second feature maps with different resolutions are obtained, and the sizes are: [4, 128, 128, 256], [4, 64, 64, 256], [4, 32, 32, 256], [4, 16, 16, 256], [4, 16, 16, 256]; Inputting five second feature maps of different resolutions of any target training image into a candidate region generation network to obtain each training detection box containing a leaf tip of any target training image; Among them, five second feature maps with different resolutions are input into the candidate region generation network, anchor pre-selection boxes of different scales are generated on the input second feature maps with different resolutions, the feature regions corresponding to the anchors are classified and regressed, the precise position information of the target is learned, and each training detection box containing the leaf tip of the target training image is obtained; Any image to be tested is an image to be tested with a red light background or multiple images to be tested with a white light background; and according to any target detection frame of the rice in the space to be tested, the step of determining the spatial coordinates of the leaf tip in the target detection frame in the preset coordinate system comprises: When any of the target detection frames is a target detection frame corresponding to a white light background image to be tested, based on color information, a rice plant body is extracted from the white light background image to be tested corresponding to the any of the target detection frames to obtain a plant extraction image corresponding to the white light background image to be tested, and the any of the target detection frames is mapped to the plant extraction image corresponding to the white light background image to be tested to obtain a target detection light mapping area of ​​the any of the target detection frames, and a Harris-Laplace corner point detector is used to extract all corner point features in the target detection light mapping area of ​​the any of the target detection frames, and a KMeans clustering algorithm is used to cluster all corner point features in the target detection light mapping area of ​​the any of the target detection frames to obtain the spatial coordinates of the leaf tip in the any of the target detection frames in the preset coordinate system; When any of the target detection frames is a target detection frame corresponding to the image to be detected under the red light background, the center point position of any of the target detection frames is determined as the spatial coordinate of the blade tip in the preset coordinate system; It also includes: determining each target detection frame corresponding to any leaf tip based on the nearest neighbor matching method of spatial position; wherein, the spatial coordinates of the leaf tips in all target detection frames are obtained, and from all the spatial coordinates, the spatial coordinates corresponding to any leaf tip are determined using the nearest neighbor matching method of spatial position to obtain the growth trajectory of any leaf tip in multiple consecutive frames.

2. The spatial rice leaf tip detection and growth trajectory generation method according to claim 1, characterized in that: Also includes: A plurality of original training images containing space rice are obtained, and data enhancement processing is performed on each of the original training images to obtain a plurality of target training images.

3. The spatial rice leaf tip detection and growth trajectory generation method according to claim 1 or 2, characterized in that: Also includes: Each leaf tip in each target training image is labeled respectively to obtain a plurality of training annotation boxes containing the leaf tips corresponding to each target training image.

4. A spatial rice leaf tip detection and growth trajectory generation system, characterized in that: include: Training module, detection module and generation module; The training module is used to: train a preset target detection model for detecting the tip of rice leaves in space based on a plurality of target training images containing rice in space, to obtain a trained target detection model; The detection module is used to: input the images of rice in the space to be detected at different times into the trained target detection model to obtain the target detection frame where the tip of the rice leaf in the space to be detected in each image to be detected is located; The generating module is used to: determine the spatial coordinates of any leaf tip of the rice to be detected in the preset coordinate system according to each target detection frame corresponding to the leaf tip of the rice to be detected, and generate the growth trajectory of the leaf tip based on all the spatial coordinates corresponding to the leaf tip; The training module includes: a first training module and a second training module; The first training module is used to: input any target training image into the preset target detection model to obtain each training detection frame containing a leaf tip in the any target training image, and obtain the loss value of the any target training image according to the training annotation frame and the training detection frame of each identical leaf tip in the any target training image, until the loss value of each target training image is obtained; The second training module is used to: optimize the network parameters of the preset target detection model based on all loss values ​​to obtain an optimized target detection model, use the optimized target detection model as the preset target detection model and return to call the first training module until the optimized target detection model meets the preset training conditions, and then determine the optimized target detection model as the trained target detection model; The preset target detection model includes: a backbone network that integrates position attention, a multi-scale feature pyramid network, and a candidate region generation network that are sequentially connected; the step of inputting any target training image into the preset target detection model in the first training module to obtain each training detection frame containing a leaf tip of any target training image includes: Inputting any target training image into the backbone network of the fused position attention to obtain four first feature maps of different resolutions of any target training image; Among them, the backbone network fused with position attention integrates position attention in the BottleNeck structure of the backbone network, and the number of input and output channels of the backbone network fused with position attention remains unchanged; the sizes of the first feature maps of four different resolutions are: [B, 128, 128, 256], [B, 64, 64, 512], [B, 32, 32, 1024], [B, 16, 16, 2048]; B is the batch size set in the experiment; Inputting four first feature maps of different resolutions of any target training image into a multi-scale feature pyramid network to obtain five second feature maps of different resolutions of any target training image; Among them, after upsampling, fusion and convolution processing of adjacent first feature maps, five second feature maps with different resolutions are obtained, and the sizes are: [4, 128, 128, 256], [4, 64, 64, 256], [4, 32, 32, 256], [4, 16, 16, 256], [4, 16, 16, 256]; Inputting five second feature maps of different resolutions of any target training image into a candidate region generation network to obtain each training detection box containing a leaf tip of any target training image; Among them, five second feature maps with different resolutions are input into the candidate region generation network, anchor pre-selection boxes of different scales are generated on the input second feature maps with different resolutions, the feature regions corresponding to the anchors are classified and regressed, the precise position information of the target is learned, and each training detection box containing the leaf tip of the target training image is obtained; Any image to be tested is an image to be tested under a red light background or multiple images to be tested under a white light background; the step of determining the spatial coordinates of the leaf tip in the target detection frame in the preset coordinate system according to any target detection frame of the rice in the space to be tested in the generation module comprises: When any of the target detection frames is a target detection frame corresponding to a white light background image to be tested, based on color information, a rice plant body is extracted from the white light background image to be tested corresponding to the any of the target detection frames to obtain a plant extraction image corresponding to the white light background image to be tested, and the any of the target detection frames is mapped to the plant extraction image corresponding to the white light background image to be tested to obtain a target detection light mapping area of ​​the any of the target detection frames, and a Harris-Laplace corner point detector is used to extract all corner point features in the target detection light mapping area of ​​the any of the target detection frames, and a KMeans clustering algorithm is used to cluster all corner point features in the target detection light mapping area of ​​the any of the target detection frames to obtain the spatial coordinates of the leaf tip in the any of the target detection frames in the preset coordinate system; When any of the target detection frames is a target detection frame corresponding to the image to be detected under the red light background, the center point position of any of the target detection frames is determined as the spatial coordinate of the blade tip in the preset coordinate system; Also included: a determination module; The determination module is used to: determine each target detection frame corresponding to any leaf tip based on the nearest neighbor matching method of the spatial position; wherein the spatial coordinates of the leaf tips in all target detection frames are obtained, and from all the spatial coordinates, the spatial coordinates corresponding to any leaf tip are determined using the nearest neighbor matching method of the spatial position, so as to obtain the growth trajectory of any leaf tip in multiple consecutive frames.

5. The spatial rice leaf tip detection and growth trajectory generation system according to claim 4, characterized in that: Also includes: Processing module; The processing module is used to obtain a plurality of original training images containing space rice, and perform data enhancement processing on each original training image to obtain a plurality of target training images.

6. A storage medium, characterized in that: The storage medium stores instructions, and when a computer reads the instructions, the computer executes the spatial rice leaf tip detection and growth trajectory generation method according to any one of claims 1 to 3.

Images