A method and robot for intelligently acquiring field crop phenotypes

Abstract

The embodiment of the present application provides a method and robot for intelligently acquiring field crop phenotypes, the method comprising: using an improved FairMOT model to acquire the category information and location information of each crop in the crop data, and generating a single crop image according to the location information; using an improved YOLOv8‑seg model to perform designated organ segmentation on the single crop image, obtaining the pixel range of the designated crop organ, and obtaining the number of pixels of the crop phenotype according to the crop phenotype to be measured; inputting the number of pixels into a calculation formula to obtain crop phenotype data. This method can effectively improve the efficiency and quality of acquiring crop phenotypic data, thereby providing better data support for breeding work.

Description

Technical Field

[0001] The present invention relates to the field of crop phenotype recognition, and in particular to a method and a robot for intelligently acquiring field crop phenotypes. Background Art

[0002] Phenotypic characteristics of crops directly reflect their performance in the actual production environment, including growth and development, physiological state, stress resistance and yield. In breeding practice, accurate evaluation of phenotypic characteristics is the basis for screening excellent varieties. By obtaining rich phenotypic data, we can gain a deep understanding of the correlation between genotype and phenotype, analyze the genetic mechanism that controls the target trait, and thus guide breeders to conduct scientific gene selection.

[0003] Existing crop phenotyping methods mainly rely on manual methods. However, manual collection methods require the collectors to be familiar with the structure and phenotype of the crops and have relevant professional knowledge. In addition, there are many crops in the field environment. Even if the collectors have relevant professional knowledge, their collection efficiency is relatively low. In addition, automated collection equipment has been widely used in the agricultural field. However, this type of equipment cannot adapt to the complex field environment, and the collection efficiency is low and the collection quality is not high.

[0004] Therefore, how to efficiently collect high-quality crop phenotypic data has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a method and robot for intelligently acquiring field crop phenotypes, which can improve the efficiency and quality of crop phenotype acquisition, thereby improving the operational efficiency of breeding work.

[0006] In a first aspect, a method for intelligently acquiring field crop phenotypes is provided, the method comprising: using an improved FairMOT model to acquire category information and position information of each crop in crop data, and generating a single crop image according to the position information, wherein the improvement measures of the FairMOT model comprise: adding a BiLevel Spatial Attention Module (BSAM) to the backbone network of the FairMOT model, and using a GS (Group Softmax) operation in a re-identification (Re-Identification, Re-ID) module of the FairMOT model; using an improved YOLOv8-seg model to perform designated organ segmentation on the single crop image to obtain a pixel range of the designated crop organ, and obtaining the number of pixels N of the crop phenotype according to the crop phenotype to be measured, wherein the improvement measures of the YOLOv8-seg model comprise: using MobileNetv4 as the backbone network of the YOLOv8-seg model, and using a distribution shift convolution (DSConv) to improve the C2f module at the neck of the YOLOv8-seg model; and obtaining crop phenotype data L according to a calculation formula.

[0007] It should be understood that this method can simultaneously obtain crop phenotypic images and size data of designated crop organs, avoiding the operation of separately obtaining two types of crop phenotypic data and improving the efficiency of obtaining crop phenotypic data.

[0008] It should be understood that the FairMOT model includes a backbone network, a detection module and a Re-ID module, wherein the backbone network includes 7 double downsampling operations, and the Re-ID module includes 1 3×3 convolution module and 1 1×1 convolution module.

[0009] In combination with the first aspect, in some implementations of the first aspect, a double-layer spatial attention module is added between the two pre-convolution modules of each deep aggregation convolution module in the backbone network of the FairMOT model. This improved method can improve the adaptability of the model to complex scenes and avoid the loss of key information; the GS operation is used to replace the SoftMax operation after the fully connected layer in the Re-ID module to alleviate the weight suppression of high-frequency categories on low-frequency categories. The position of a single crop in the image is obtained by improving the FairMOT model, and the category information of the crop is obtained at the same time.

[0010] In combination with the first aspect, in some implementations of the first aspect, distribution shift convolution is used to replace two ordinary convolution modules in the C2f module at the neck of the YOLOv8-seg model to reduce the number of parameters of the model and improve computational efficiency.

[0011] It should be understood that the YOLOv8-seg model includes 8 C2f modules, of which 4 C2f modules are located at the neck of the YOLOv8-seg model. Each C2f module contains two ordinary convolution modules and two bottleneck structures (BottleNeck). The two ordinary convolutions are located at the start and end of the C2f module respectively.

[0012] In combination with the first aspect, in some implementations of the first aspect, the method includes: using a visible light camera to collect crop data, and using a depth camera to record a distance M between the crop and the depth camera.

[0013] Optionally, a depth camera may be used to capture a crop video, while recording a distance M between the crop and the depth camera.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: screening the crop data to eliminate duplicate and low-quality crop data.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the calculation formula is:

[0016] ,

[0017] Among them, FOV is the camera field of view, b is the bias term, and this application obtains b as 1.6 and FOV as 110 through data fitting. In actual tasks, only the number of pixels N of the crop organ and the distance M between the crop and the depth camera need to be input to obtain the actual size of the crop organ, which improves the efficiency of obtaining crop phenotypic data.

[0018] In a second aspect, a field crop phenotype intelligent acquisition robot is provided, the robot comprising: a collection robot, an acquisition module and an operation platform, wherein the acquisition module can execute the method in any one of the implementation modes in the first aspect.

[0019] In combination with the second aspect, in certain implementations of the second aspect, the collection robot includes a chassis, which is a four-axle four-wheel drive vehicle. The four-axle four-wheel drive vehicle can perform Ackerman motion and lateral movement, thereby ensuring that the collection robot has sufficient maneuverability to adapt to the complex environment in the field and improve the efficiency of crop data collection.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the collection robot includes an expansion support structure, the expansion support structure includes a simple electrical cabinet and an expansion rod, and the expansion rod is fixedly installed above the simple electrical cabinet. The expansion support structure provides a stable installation platform for multiple modules to ensure smooth crop data collection.

[0021] In conjunction with the second aspect, in certain implementations of the second aspect, the collection robot includes a control and communication system, which includes an internal part and an external part. The internal part is equipped with a vehicle controller, which is responsible for the motion control and communication of the vehicle, and the internal part is arranged in the chassis; the external part is arranged in a simple electrical cabinet, which is responsible for the control of the entire collection robot, and receives and processes corresponding instructions issued by the vehicle controller. The internal part and the external part cooperate with each other to realize the motion and communication control of the entire collection robot, thereby ensuring the smooth progress of crop data collection.

[0022] In combination with the second aspect, in some implementations of the second aspect, the acquisition robot includes a acquisition module, the acquisition module includes a visible light camera and a depth camera, and the acquisition module is installed on the expansion support structure.

[0023] In combination with the second aspect, in some implementations of the second aspect, the operating platform uses a front-end and back-end separated development architecture, the back-end framework is Django, the front-end framework is Vue, and the database is MySQL.

[0024] Compared with the prior art, the beneficial effects of the present application are as follows: the present application generates a single crop image by improving the FairMOT model, identifies its category at the same time, and then uses the improved YOLOv8-seg model to perform designated crop organ segmentation on the single crop image, and finally obtains the crop phenotypic data through a calculation formula; in addition, in conjunction with the crop phenotypic acquisition robot, crop data can be collected stably and efficiently, and crop phenotypic data can be quickly and accurately acquired, including crop phenotypic images and crop organ size data, thereby avoiding the operation of separately acquiring two types of crop phenotypic data and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the structure of a robot for intelligently acquiring field crop phenotypes provided in an embodiment of the present application.

[0026] Figure 2 This is a flow chart of a method for intelligently acquiring field crop phenotypes provided in an embodiment of the present application.

[0027] Figure 3 It is a schematic diagram of the FairMOT model structure.

[0028] Figure 4 It is a schematic diagram of the module structure of the double-layer spatial attention mechanism.

[0029] Figure 5 It is a schematic diagram of an improved FairMOT model structure provided in an embodiment of the present application.

[0030] Figure 6 This is a schematic diagram of the C2f module structure.

[0031] Figure 7 It is a schematic diagram of the structure of a collection robot provided in an embodiment of the present application.

[0032] Figure 8 It is a schematic diagram of the structure of a control and communication system provided in an embodiment of the present application.

[0033] Fig. 9 This is a flow chart of obtaining crop phenotypic data using an operating platform provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0035] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0036] Crop phenotypic data is of great significance to crop breeding. Rich phenotypic data can help researchers gain a deeper understanding of the correlation between genotype and phenotype, analyze the genetic mechanism that controls the target trait, and thus guide breeders to make scientific gene selection. However, the existing methods for obtaining crop phenotypic data often have problems such as low efficiency, high cost, and difficulty in adapting to complex field environments, which restricts breeding research.

[0037] The embodiments of the present application provide a method and robot for intelligently acquiring field crop phenotypes, which can improve the efficiency and quality of acquiring crop phenotypic data, thereby accelerating the breeding process and improving breeding operation efficiency.

[0038] The technical solution in this application is described below in conjunction with the accompanying drawings.

[0039] Figure 1 1 is a schematic diagram of a field crop phenotype intelligent acquisition robot 100 provided in an embodiment of the present application. The robot 100 includes: a collection robot 110, an acquisition module 120, and an operating platform 130. The acquisition module 120 can execute any method in the embodiments described below. The robot 100 can also include other subsystems, and each subsystem also includes one or more smaller subsystems.

[0040] Optionally, each subsystem in the robot 100 can communicate with each other via wired or wireless means, or through a specially configured communication system, which is not limited in this embodiment of the present application.

[0041] Figure 2 It is a flow chart of a method 200 for intelligently acquiring field crop phenotypes provided in an embodiment of the present application. The method 200 can be executed by the acquisition module 120. The method 200 can include steps S201, S202 and S203.

[0042] Step S201, using the improved FairMOT model to obtain the category information and location information of each crop, and generating a single crop image according to the location information; wherein the improvement measures of the FairMOT model include: adding a BSAM attention mechanism module in its backbone network and using a GS operation in its Re-ID module.

[0043] Optionally, the position information of each crop includes: the center coordinates of the crop in the image and the length and width of the detection frame, or the coordinates of the upper left point and the lower right point of the detection frame of each crop in the image.

[0044] It should be understood that Figure 3 As shown in the figure, the FairMOT model includes a backbone network, a detection module and a Re-ID module. The backbone network includes 7 double downsampling operations, and the Re-ID module includes a 3×3 convolution module and a 1×1 convolution module.

[0045] In some examples, a double-layer spatial attention module is added between the two pre-convolution modules of each deep aggregation convolution module in the backbone network of the FairMOT model to improve the adaptability of the FairMOT model to complex scenes and avoid the loss of key information, thereby improving the FairMOT model's ability to locate and identify crops; the GS operation is used to replace the SoftMax operation after the fully connected layer in the re-identification module to alleviate the weight suppression of high-frequency categories on low-frequency categories. This improves the recognition accuracy of the FairMOT model for crop categories.

[0046] It should be understood that the attention mechanism is to calculate the importance of input features so that the automatic model focuses on the features most relevant to the current task while suppressing the interference of irrelevant or noise features. The BSAM module structure is as follows Figure 4 As shown in the figure, it includes a bi-level routing module (Bi-Level Routing Attention Module) and a spatial attention module (Spatial Attention Module). The BSAM module enables the model to show stronger robustness in scenes with occlusion, complex backgrounds, and feature interference, thereby enhancing the generalization ability of the model, which is suitable for the needs of crop detection in complex field environments, and can efficiently track and detect individual crop plants in this environment. The GS operation is aimed at the imbalance problem of crop phenotypic feature categories. These feature categories are divided into several independent category groups according to their frequencies in the training data set, and the Softmax operation is performed on each group. This grouping mechanism can place categories with similar frequencies in the same group, allowing them to compete within the group, thereby effectively alleviating the problem of weight suppression of high-frequency categories on low-frequency categories. According to the above measures, the improved FairMOT model structure is as follows Figure 5 shown.

[0047] Step S202, using the improved YOLOv8-seg model to segment the single crop image into designated crop organs, obtain its pixel range, and obtain the number of pixels N of the corresponding phenotype according to the crop phenotype to be measured; wherein the improvement measures of the YOLOv8-seg model include: using the MobileNetV4 model as its backbone network, and using the DSConv module to improve the C2f module of its neck.

[0048] It should be understood that MobileNetV4 is a lightweight and efficient convolutional neural network model designed for resource-constrained scenarios and suitable for use in mobile devices or edge computing environments. The embodiment of this application uses MobileNetV4 as the backbone network of the YOLOv8-seg model, which not only greatly reduces the number of parameters and computational complexity, but also maintains a high accuracy in feature extraction capabilities. Distribution Shift Convolution (DSConv) is an efficient variant optimized for traditional convolutional layers. Its core design concept is to reduce memory usage and increase computing speed while maintaining output performance comparable to standard convolution operations.

[0049] refer to Figure 6 , the C2f module includes two ordinary convolution modules and two bottleneck structures, and the two ordinary convolutions are located at the start and end positions of the C2f module respectively. In a possible implementation, the embodiment of the present application uses DSConv to replace the two ordinary convolutions located at the start and end positions in the C2f module of the neck of the YOLOv8-seg model, further reducing the number of parameters and computational complexity.

[0050] In a possible implementation, the number of pixels in each row and column is calculated and counted through the binary mask output after segmentation, and an index is established to obtain the number of pixels N of the required crop phenotype.

[0051] Step S203, input the number of pixels N into the calculation formula to obtain the crop phenotype number L.

[0052] In some examples, the method 200 for intelligently acquiring field crop phenotypes further includes: acquiring crop data. In a possible implementation, a visible light camera is used to shoot a crop video, and a depth camera is used to record a distance M between the crop and the depth camera.

[0053] Optionally, a crop video is captured using a depth camera, and a distance M between the crop and the depth camera is recorded.

[0054] In some examples, the method 200 for intelligently acquiring field crop phenotypes further includes: screening the crop data to remove duplicate data or data of low quality.

[0055] In some examples, crop phenotyping data is calculated as:

[0056] ,

[0057] Among them, FOV is the field of view of the camera, and b is the bias term. In actual tasks, you only need to input the number of pixels N of the crop organ and the distance M between the crop and the depth camera to get the actual size of the crop organ, which improves the efficiency of obtaining crop phenotypic data.

[0058] In a possible implementation, the depth value of the center pixel of the improved detection frame generated based on FairMOT is selected as the distance between the depth camera and the crop, and the depth value M of the center pixel is read from the depth map. For the collected visible light data of the crop, the number of pixels N of its phenotypic data in the visible light image is recorded by the segmentation method. Assuming that the crop phenotypic data is L, the camera field of view is FOV, and the camera resolution is 1920×1080. According to the relationship between the camera viewing angle and the image range and the pixel ratio conversion, the crop phenotypic data can be obtained.

[0059] In actual tasks, the depth information between the depth camera used for recording on the robot and the crop is obtained, and then the number of crop organ pixels obtained by the crop segmentation algorithm is combined with the camera's own field of view angle parameters to obtain the actual crop organ size phenotype.

[0060] Figure 7 This is a schematic diagram of a collection robot 110 provided in an embodiment of the present application. In some examples, the collection robot includes a chassis 111. In one possible implementation, the chassis 111 is a four-axle four-wheel drive vehicle.

[0061] In a possible implementation, the skeleton structure of the vehicle chassis 111 is mainly constructed of aluminum alloy plates, with a mass of no more than 27kg, dimensions of 495×360×320 (mm), a wheelbase of 350mm, a track of 250mm, and is equipped with 5.5-inch (140mm) hard deep-pattern off-road tires. The vehicle chassis 111 has both the Ackerman motion mode of the four-axis four-wheel drive and the lateral movement mode. In the four-axis four-wheel drive Ackerman mode, it has the advantages of stability, no side slip, and a small turning radius of the Ackerman corner turning type car, which ensures that the collection robot has sufficient power and ensures that it has good off-road obstacle crossing ability in the fields. With the lateral movement mode, the collection robot has stronger flexibility and maneuverability, which is conducive to better adapting to various complex environments. At the same time, it improves the robot's self-rescue ability when encountering difficulties in actual use scenarios, and effectively ensures that the robot has a certain ability to escape from difficulties in extreme conditions and environments.

[0062] Optionally, the chassis 111 of the collection robot 110 is a quadruped robot, or a tracked structure.

[0063] In some examples, the collection robot 110 includes an expansion support structure 112 , and the expansion support structure 112 includes a simple electrical cabinet 1121 and an expansion rod 1122 , and the expansion rod 1122 is fixed above the simple electrical cabinet 1121 .

[0064] Optionally, the simple electrical cabinet 1121 is composed of multiple brackets of different lengths, and the expansion rod 1122 is installed above the simple electrical cabinet 1121. The side camera brackets are installed at appropriate heights on both sides of the expansion rod 1122, and their heights and angles can be flexibly adjusted according to actual use requirements. Four 45° double-inner bevel brackets are set at the connection between the two, and special connecting angle pieces and nuts are used to reinforce the expansion rod 1122 from the front, back, left and right sides to reduce the impact of the bumps of the collection robot in the actual field movement and the impact on the camera and sensor device on the expansion rod 1122 after being magnified by the structure layer by layer.

[0065] In some examples, the collection robot 110 also includes a control and communication system 113, referring to Figure 8 The control and communication system 113 includes an internal part 1131 and an external part 1132 , and these two parts communicate through the CAN reserved on the chassis 111 .

[0066] Optionally, the internal part 1131 is equipped with a vehicle controller with a main frequency of 168MHz as the vehicle motion control center. The controller supports hardware floating-point acceleration function, supports CAN bus standard communication protocol, has a CAN bus communication interface, and has both wired and wireless communication functions. The external part 1132 is installed in the lower simple electrical cabinet 1121 of the expansion support structure 112, which is responsible for the control of the overall acquisition robot, receiving and processing the corresponding instructions issued by the vehicle controller. In a possible implementation, the external part 1132 may include an industrial computer, a wireless bridge vehicle-mounted part, a video recording module, and an energy conversion device.

[0067] In some examples, the collection robot 110 also includes a collection module 114 , which includes a visible light camera and a depth camera, and is installed on the extended support structure 112 of the collection robot 110 .

[0068] In some examples, the robot 100 also includes an operating platform 130, which is developed in the form of a web page, and users can directly access and use functions through a browser. The operating platform 130 adopts a development architecture with front-end and back-end separation, and the back-end uses Django and MySQL as the development framework and database; the front-end uses the Vue framework, combined with Element-UI to achieve intuitive and friendly page rendering.

[0069] Optionally, the operating platform 130 supports multiple user roles, including ordinary users, administrators, and super administrators. Ordinary users can access phenotypic data and complete basic operations; administrators are responsible for managing user accounts and assigning permissions; super administrators have global system configuration and monitoring permissions. The strict role-based permission control mechanism ensures the security and standardization of the operating platform 130. In terms of deployment, the operating platform 130 uses Docker containerization technology to achieve environmental isolation of components, ensuring that each module runs independently and improving stability and scalability.

[0070] In one possible implementation, reference Fig. 9 The steps for the operating platform 130 to obtain crop phenotypic data are as follows:

[0071] Step 1: Enter the system login interface of the operating platform and enter the user name and password to log in;

[0072] Step 2: Enter the phenotypic analysis module and search for the phenotypic video that is automatically uploaded for analysis;

[0073] Step 3: View the video with phenotypic test results. You can select "Download csv file" to obtain various phenotypic data indicators of the crops in the video.

[0074] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A method for intelligently acquiring field crop phenotypes, characterized in that: The method comprises: The improved FairMOT model is used to obtain the category information and location information of each crop in the crop data, and a single crop image is generated according to the location information; Among them, the improvement measures of the FairMOT model include: adding a double-layer spatial attention module between the two pre-convolution modules of each deep aggregation convolution module in the backbone network of the FairMOT model, and replacing the SoftMax operation after the fully connected layer with a grouped Softmax (GS) operation in the re-identification (Re-ID) module of the FairMOT model; Using the improved YOLOv8-seg model to perform designated organ segmentation on the single crop image, obtain the pixel range of the designated crop organ, and obtain the pixel number N of the crop phenotype according to the crop phenotype to be measured; Among them, the improvement measures of the YOLOv8-seg model include: using MobileNetv4 as the backbone network of the YOLOv8-seg model, and using distribution shift convolution (DSConv) to replace two ordinary convolution modules located at the start and end positions in the C2f module at the neck of the YOLOv8-seg model; The crop phenotype data L is obtained according to the calculation formula.

2. The method for intelligently acquiring field crop phenotypes according to claim 1, characterized in that: The method comprises: using a visible light camera to shoot the crop data, and using a depth camera to record a distance M between the crop and the depth camera.

3. The method for intelligently acquiring field crop phenotypes according to claim 1, characterized in that: The method further comprises: screening the crop data to remove duplicate crop data or low-quality crop data.

4. The method for intelligently acquiring field crop phenotypes according to claim 1, characterized in that: The calculation formula is: ， Among them, FOV is the field of view of the camera, and b is the bias term.

5. A robot for intelligently acquiring field crop phenotypes, characterized in that: The robot (100) comprises: a collection robot (110), an acquisition module (120) and an operation platform (130), wherein the acquisition module (120) can execute the method according to any one of claims 1 to 4.

6. The field crop phenotype intelligent acquisition robot according to claim 5, characterized in that: The collection robot (110) comprises: A chassis (111), wherein the chassis (111) is a four-axle four-wheel drive vehicle, and the four-axle four-wheel drive vehicle includes two motion modes: Ackerman and lateral movement; An expansion support structure (112), the expansion support structure (112) comprising a simple electrical cabinet (1121) and an expansion rod (1122), the expansion rod (1122) being fixedly mounted on the simple electrical cabinet (1121); A control and communication system (113), the control and communication system (113) comprising an internal part (1131) and an external part (1132), the internal part (1131) being equipped with a vehicle controller, the internal part (1131) being arranged in the chassis (111), and the external part (1132) being arranged in the simple electrical cabinet (1121).

7. The field crop phenotype intelligent acquisition robot according to claim 6, characterized in that: The acquisition robot (110) further comprises an acquisition module (114), wherein the acquisition module (114) comprises a visible light camera and a depth camera, and the acquisition module (114) is mounted on the expansion support structure (112).

8. The field crop phenotype intelligent acquisition robot according to claim 5, characterized in that: The operating platform (130) adopts a front-end and back-end separation development architecture, with the back-end framework being Django, the front-end framework being Vue, and the database being MySQL.

Images