Plant multi-modal phenotype collection assembly line device and method

Abstract

The invention provides a plant multi-mode phenotype collection assembly line device and method, and is applied to the technical field of plant assembly line phenotype collection, a planting column is provided with a plurality of columns of roller structure conveying lines which are arranged in parallel, and the roller structure conveying lines carry tested plants with signboards; the conveying column is composed of a conveying inlet column, a conveying outlet column and a fluorescent darkroom channel, and an inlet of the conveying inlet column is provided with an inlet column detection unit used for identifying a signboard of a detected plant; the fluorescent darkroom channel is connected with the acquisition column, and the acquisition column is sequentially integrated with the fluorescent imaging darkroom, the spectral imaging box body and the 3D imaging box body according to the transmission direction so as to respectively perform plant fluorescent image acquisition, plant spectral image acquisition and plant multi-view image acquisition; and the display unit is used for visualizing an operation interface, monitoring the equipment state in real time and displaying the imaging data of the detected plant and the transmission position of the detected plant. According to the invention, high-throughput automatic collection of plant multi-modal phenotype data can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of plant pipeline phenotype acquisition, and particularly relates to a plant multi-modal phenotype acquisition pipeline device and method. Background Art

[0002] The accurate and rapid acquisition of plant phenotype data is the main data source for constructing plant phenotype big data, providing basic data support services for crop digital breeding, digital cultivation, and intelligent management research and application of agricultural production. Currently, the monitoring and analysis means of plant phenotypes are gradually developing towards the direction of integration, scale, multi-scale, and high-throughput.

[0003] According to different scales, the current phenotype acquisition platforms include: remote sensing satellite platforms, low-altitude unmanned aerial vehicle phenotype platforms, field fixed-rail phenotype platforms, phenotype vehicles, phenotype robots, portable phenotype platforms, and pipeline-type phenotype platforms. The pipeline-type phenotype platform is one of the main technical means for continuous monitoring, refinement, and high-throughput acquisition of plant single-plant scale phenotypes. A well-designed pipeline-type phenotype platform should have technical characteristics and advantages such as continuous monitoring during the plant growth period, high precision, automation, multi-dimension, and multi-modal synchronous high-throughput acquisition.

[0004] Existing pipeline-type phenotype platforms need to integrate multiple types of sensors to meet the data acquisition requirements. However, the differences in imaging conditions and data synchronization of different sensors result in limited plant phenotype acquisition efficiency and throughput of the pipeline platform. Summary of the Invention

[0005] The present invention provides a plant multi-modal phenotype acquisition pipeline device and method to solve the defect of limited plant phenotype acquisition efficiency and throughput of the pipeline platform in the prior art, and to achieve high-throughput automation acquisition of plant multi-modal phenotype data.

[0006] The present invention provides a plant multi-modal phenotype acquisition pipeline device, comprising: a planting column, a conveying column and an acquisition column, wherein: the planting column is provided with a plurality of rows of roller structure conveying lines arranged in parallel, and the roller structure conveying lines carry the plants to be measured with identification plates; the conveying column consists of a conveying inlet column, a conveying outlet column and a fluorescence darkroom passage. Among them, an inlet column detection unit is provided at the entrance of the conveying inlet column for identifying the identification plate of the plant to be measured; the planting column is connected to the conveying inlet column and the conveying outlet column through a steering structure, and the conveying outlet column is connected to the fluorescence darkroom passage through a steering structure; the fluorescence darkroom passage is connected to the acquisition column, and the acquisition column is sequentially integrated with a fluorescence imaging darkroom, a spectral imaging box body and a 3D imaging box body in the conveying direction to respectively perform plant fluorescence image acquisition, plant spectral image acquisition and plant multi-view image acquisition; the planting column, the conveying column and the acquisition column are respectively driven by independent drive motors, and limit detection mechanisms are provided at the connection between the conveying inlet column and the planting column, the connection between the planting column and the conveying outlet column, the connection between the conveying outlet column and the fluorescence darkroom passage, and the connection between the fluorescence darkroom passage and the acquisition column. The limit detection mechanism is used to monitor the conveying position of the plant to be measured; the drive motors and the limit detection mechanisms are centrally controlled by a programmable logic controller of the central control unit, and the central control unit communicates with a display unit to feedback data in real time. The display unit is used for a visual operation interface, real-time monitoring of the device status, displaying the imaging data of the plant to be measured and the conveying position of the plant to be measured, wherein the imaging data of the plant to be measured and the conveying position of the plant to be measured are matched with the identification plate.

[0007] According to the plant multi-modal phenotype acquisition pipeline device provided by the present invention, the 3D imaging box body comprises: an electric switch door which is respectively installed on the front side and the rear side of the 3D imaging box body; a rotary turntable, on the rotating arm of which a plurality of RGB image sensors are installed to form a multi-view image array sensor unit for collecting multi-view images of the plant to be measured; a lifting guide rail, on which at least one RGB image sensor is installed for collecting top-view images of the plant to be measured; a lighting lamp which is respectively installed around the inner wall and at the top of the inner wall of the 3D imaging box body; wherein, the electric switch door, the rotary turntable, the lifting guide rail and the lighting lamp are electrically connected to the central control unit through a 3D imaging controller, and the central control unit respectively controls the opening and closing state of the electric switch door, the rotation angle of the rotary turntable, the lifting height of the lifting guide rail and the brightness of the lighting lamp. The RGB image sensors are communicatively connected to the central control unit, and the central control unit is further used for collecting the multi-view images and the top-view images.

[0008] A plant multi-modal phenotype acquisition pipeline device provided by the present invention, the fluorescence imaging darkroom includes: a fluorescence imaging box body and a dark treatment channel; electric switch doors are installed on the front side and the rear side of the fluorescence imaging box body, a lifting slide rail is installed on the top of the fluorescence imaging box body, and a fluorescence light source and a fluorescence camera are installed below the lifting slide rail; wherein, the electric switch door, the fluorescence light source, the fluorescence camera and the central control unit are electrically connected, and the central control unit is further used to control the opening and closing state of the electric switch door, the opening and closing state of the fluorescence light source, and the fluorescence image acquisition of the fluorescence camera; the dark treatment channel is used to form a dark environment with the fluorescence imaging box body after the electric switch door is closed.

[0009] A plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: a water and fertilizer integrated irrigation unit, the water and fertilizer integrated irrigation unit is located at one end of the conveying incoming column, and the water and fertilizer integrated irrigation unit includes: a fertilizer application tank, a fertilizer suction pump, an adjustable water outlet pipe, a flow meter and a solenoid valve; wherein, the flow meter and the solenoid valve are communicatively connected with the central control unit.

[0010] A plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: a weighing unit, the weighing unit is installed at one end of the acquisition column, and the weighing unit includes a plurality of weighing sensors, and the weighing sensors are respectively located on the four corner support columns of the weighing conveyor line of the acquisition column; wherein, the weighing sensors are electrically connected with the central control unit.

[0011] A plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: a plant growth supplementary lighting unit, the plant growth supplementary lighting unit is suspended above the planting column, and a plurality of controllable and adjustable supplementary lighting units are installed according to the area size; the supplementary lighting unit is composed of a multi-node full-spectrum supplementary light lamp, a supplementary light lamp suspension rod, a lifting system and a supplementary light controller, and the lifting system and the full-spectrum supplementary light lamp are electrically connected with the supplementary light controller; the supplementary light controller is used to control the on / off of the full-spectrum supplementary light lamp, the light quantum size of the full-spectrum supplementary light lamp, the spectral channel of the full-spectrum supplementary light lamp, and the supplementary light controller is further used to control the height of the supplementary lighting unit from the measured plant; the supplementary lighting unit covers the planting column in areas and is used to supplement light to the measured plants on the planting column as needed.

[0012] A plant multi-modal phenotype acquisition pipeline device provided according to the present invention, the device further includes: a calibration unit, including: an RGB color calibration board and a first image pixel registration board located on both sides of the center of the conveyor line in the 3D imaging box body, wherein the RGB color calibration board is a square board surface, and is sprayed with a 24-color standard color card RGB image calibration board pattern; a spectral calibration board and a second image pixel registration board located on both sides of the center of the conveyor line in the spectral imaging box body, wherein the spectral calibration board is a square board surface, and is sprayed with a spectral calibration board pattern with different reflectivities; a fluorescence calibration board and a third image pixel registration board located on both sides of the center of the conveyor line in the fluorescence imaging box body, wherein the fluorescence calibration board is a square board surface, and is sprayed with a spectral calibration board pattern with different reflectivities; the first image pixel registration board, the second image pixel registration board, and the third image pixel registration board are all square board surfaces, and are sprayed with an image pixel registration pattern composed of a plurality of black dots.

[0013] A plant multi-modal phenotype acquisition pipeline device provided according to the present invention, the central control unit is further configured to: acquire a multi-view image sequence of the measured plant collected by the 3D imaging box body; determine 3D point cloud data based on the multi-view image sequence through a multi-view reconstruction algorithm, wherein the 3D point cloud data includes: position points and vertex colors; perform point cloud restoration based on the position points and vertex colors of the 3D point cloud data and a preset image pixel registration board through a point cloud color detection algorithm to obtain reconstructed point cloud data, wherein the size ratio of the reconstructed point cloud data to the measured plant is the same; multiply the reconstructed point cloud data by an image color correction matrix generated based on the RGB color calibration board to obtain corrected point cloud data, wherein the color space of the corrected point cloud data is the same as that of the measured plant.

[0014] The present invention also provides a method for a plant multi-modal phenotype acquisition pipeline, comprising the following steps: conveying a measured plant with an identification plate through a roller structure conveyor line arranged in multiple parallel columns of a planting column; when the measured plant enters the conveying-in column of the conveying column, using an in-column detection unit to identify the identification plate; turning the measured plant from the planting column to the conveying-in column, the conveying-out column, and the fluorescence darkroom channel in sequence through a turning structure, and finally conveying it to the acquisition column; sequentially performing plant fluorescence image acquisition, plant spectral image acquisition, and plant multi-view image acquisition on the measured plant in the acquisition column in the conveying direction; wherein, the planting column, the conveying column, and the acquisition column are respectively driven by independent driving motors; at the connection between the planting column and the conveying-in column, the connection between the planting column and the conveying-out column, the connection between the conveying-out column and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition column, a position-limiting detection mechanism is used to monitor the conveying position of the measured plant in real time; the driving motors and the position-limiting detection mechanism are centrally controlled by a programmable logic controller of a central control unit, and real-time data is fed back to a display unit, and through the visual operation interface of the display unit, the device state is monitored in real time, the imaging data of the measured plant and the conveying position of the measured plant are displayed, wherein the imaging data of the measured plant and the conveying position of the measured plant are matched with the identification plate.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the method for the plant multi-modal phenotype acquisition pipeline as described in any one of the above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for the plant multi-modal phenotype acquisition pipeline as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, and when the computer program is executed by a processor, the method for the plant multi-modal phenotype acquisition pipeline as described in any one of the above is implemented.

[0018] The plant multi-modal phenotype acquisition pipeline device and method provided by the present invention realize the full-process automatic transmission of the measured plants from planting to multi-modal data acquisition through the modular pipeline design of the planting column, the conveying column and the acquisition column, and in combination with the multi-column parallel roller conveyor line, the independent drive motor and the steering structure; the acquisition column is successively integrated with a fluorescence imaging darkroom, a spectral imaging box body and a 3D imaging box body, and through the coordinated control of the limit detection mechanism and the central control unit, it accurately matches the requirements of different imaging conditions and synchronously completes the high-throughput acquisition of morphological, spectral and physiological phenotype data; at the same time, based on the identification plate recognition and real-time position tracking technology, it ensures the unique association between the data and the plants, and centrally monitors the device status and data stream through the visual interface, significantly improving the acquisition efficiency and data accuracy. Brief Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the attached drawings required for use in the description of the embodiments or the prior art one by one. Obviously, the attached drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other attached drawings can also be obtained based on these attached drawings.

[0020] Figure 1 It is a schematic diagram of the pipeline device provided by the present invention.

[0021] Figure 2 It is a schematic diagram of the conveying structure of the pipeline provided by the present invention.

[0022] Figure 3 It is a schematic diagram of the distribution and connection of each unit of the pipeline platform provided by the present invention.

[0023] Figure 4 It is a schematic diagram of the structure of the plant growth supplementary lighting unit provided by the present invention.

[0024] Figure 5 It is a connection structure diagram of the central control unit provided by the present invention.

[0025] Figure 6 It is a schematic diagram of the distribution and mounting structure of the phenotypic sensors inside the imaging box body provided by the present invention.

[0026] Figure 7 It is a schematic diagram of the multi-modal phenotype data fusion and calibration structure provided by the present invention.

[0027] Figure 8 It is a schematic diagram of the arrangement rule of plant samples in the planting column of the pipeline provided by the present invention.

[0028] Figure 9 It is a schematic diagram of the operation process of the sample entry column pipeline provided by the present invention.

[0029] Figure 10 It is a schematic diagram of the operation process of the sample water and fertilizer perfusion pipeline provided by the present invention.

[0030] Figure 11 It is a schematic diagram of the operation process of the fluorescence data acquisition pipeline of the present invention.

[0031] Figure 12 It is a flow chart of the multi-modal data fusion and phenotype analysis algorithm provided by the present invention.

[0032] Figure 13 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed implementation manners

[0033] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] In order to improve the acquisition efficiency and acquisition throughput of the phenotype platform, different imaging boxes are usually set up and different types of phenotype sensors are mounted. Therefore, how to effectively overcome the imaging requirements of different imaging units, improve the acquisition efficiency, standardize, with high precision, and synchronously acquire the phenotype data of different types of sensors is a technical difficulty. The original data obtained based on the phenotype platform includes different types of data such as RGB images, spectral images, thermal infrared images, point clouds, and fluorescence. How to achieve multi-modal data edge-side fusion and automatically analyze the temporal phenotype data is another technical difficulty. Therefore, constructing a pipeline-type phenotype platform device, system, and phenotype analysis method to provide phenotype facility equipment for precise planting control management and high-throughput phenotype automatic acquisition of protected plants has application requirements and market prospects.

[0035] The main technical links of the plant pipeline phenotype acquisition platform include: light temperature, water and fertilizer management of plant pipeline facility planting, automatic transmission of plant pipeline, layout of sensor and imaging unit boxes, high-temporal synchronous control acquisition, and multi-modal phenotype data fusion and analysis. The main technical progress and defects are summarized as follows: In the related art, the construction of an automated collection device includes a grasping device, a conveyor, an information collection device, and a control unit to achieve automated collection of plants. A bracket is fixedly connected to the conveyor, and an information collection device is installed on the bracket. The information collection device includes a camera, which can be a visible light camera, an infrared camera, or a binocular camera. The information collection device may also include sensors such as an imaging spectrometer, a fluorescence imager, and / or a lidar scanner, etc., for obtaining phenotypic data such as the number of leaves, leaf length and width, and leaf inclination of the target potted plant. However, the defect of the above technology is that it does not provide technical solutions or methods for the spatial layout, synchronous collection, and phenotypic analysis of sensors, and only gives technical descriptions in aspects such as conveying.

[0036] In the related art, a plant phenomics information acquisition system is provided, including: a box body, a transmission device, a control device, and multiple sensors arranged at predetermined positions on the track, including a radar device, an optical imaging device, and a fluorescence imaging device. A robotic arm is arranged inside the box body to adjust the acquisition angle of the sensor device through the robotic arm. However, the defect of the above technology is that it does not describe the coordination of each sensor, does not describe the data processing method, and there is no corresponding acquisition system.

[0037] In the related art, by setting a circular arm and a motor driver, and installing multiple acquisition sensors on the circular arm, including one or more of an RGB camera, a hyperspectral camera, a multispectral camera, a depth sensor, and a thermal infrared camera of a small lidar, multi-view data acquisition of plant phenotypic data is achieved. However, the defect of the above technology is that it does not explain how to analyze and process the data, nor does it explain the data fusion method.

[0038] In the related art, an information collection device is driven by a rotating device, and a camera sensor is installed on the collection device. A checkerboard calibration board is arranged on the curtain to collect multi-view images of the plant, and based on the multi-view images, the plant phenotypic index is estimated by using the method of average value. However, the defect of the above technology is that the estimation method based on the average value has a large error.

[0039] In the related art, by designing a bent movable rod and a workbench, a slide rail and a lift camera sensor are installed on the bent movable rod, and a self-rotating stepping motor is installed on the workbench to achieve the acquisition of three-dimensional phenotypic data of the rotating plant. However, the defect of the above technology is that it does not provide a substantial technical solution for the post-processing of the data, and the main work is in the aspect of data collection.

[0040] In terms of phenotypic data fusion and processing, in the related technologies, by setting up two teams of depth cameras and multispectral cameras, the fusion of plant 3D spectral data is achieved, providing a technical solution for plant multispectral point cloud fusion from the cabinet design to the sensor layout and the data fusion method. Its technical defect is that only the data on both sides of the plant are acquired, and it is impossible to avoid the problem of data loss caused by occlusion.

[0041] The pipeline-type plant multi-modal phenotypic platform should have technical characteristics such as plant facility cultivation management control, continuous monitoring during the growth period, high precision, automation, multi-dimension, and high-throughput acquisition. Among them, the plant facility cultivation management Internet of Things, the pipeline transmission structure, the sensor layout, the imaging cabinet design, and the data acquisition system and the phenotypic analysis system should be interconnected with each other. Therefore, in the embodiments of the present invention, by optimizing the pipeline acquisition device and designing the sensor layout, the synchronous acquisition of data from different types of sensors is realized, and based on the physical information and spatial layout of the sensors, a precise planting control management, a data acquisition and fusion method, and a phenotypic analysis method for facility plants are constructed, providing a complete set of technical solutions for realizing fully automated plant multi-modal phenotype acquisition and automated analysis.

[0042] The purpose of the present invention is to construct a plant multi-modal phenotypic acquisition pipeline device and method. Based on this device and method, the automatic control of light temperature, water and fertilizer for pipeline cultivation of facility plants is realized, the high-throughput and automated acquisition of plant multi-modal phenotypic data, the registration and fusion of multi-modal phenotypic data acquisition terminals, and the automatic analysis of phenotypic indicators are specifically included: A pipeline-type transmission device and a pipeline cultivation light temperature, water and fertilizer control system are built to realize the automatic cultivation management, transmission and acquisition of multiple samples; a multi-cabinet multi-modal sensor acquisition unit is designed to acquire plant phenotypic data of different types of sensors in a high-throughput and automated manner; an automated phenotypic acquisition system is constructed, which can collect in an orderly and collaborative manner fully automatically according to the operation process without manual intervention; a method for the fusion of plant three-dimensional multi-modal phenotypic acquisition terminals and phenotypic analysis is constructed to realize the reconstruction of plant three-dimensional phenotypes, the registration and alignment fusion of multi-modal data such as point clouds, spectra, and fluorescence, the phenotypic analysis at the plant scale, and the phenotypic analysis at the organ scale. Based on the embodiments of the present invention, high-precision and multi-dimensional phenotypic indicators of plants can be accurately and quickly obtained without manual destructive sampling.

[0043] The present invention provides a plant multi-modal phenotypic acquisition pipeline device and method, which includes four parts: a pipeline phenotypic platform device, a multi-modal sensor imaging unit and an imaging cabinet, a platform control acquisition system, and a method for the fusion of plant three-dimensional multi-modal phenotypic acquisition terminals and phenotypic analysis.

[0044] Reference Figure 1 , Figure 1 is a schematic diagram of the pipeline device provided by the present invention, as Figure 1As shown in the figure, it includes: 1-1 pipeline transmission structure; 1-2 multi-view imaging box; 1-3 spectral imaging box; 1-4 dark treatment channel; 1-5 fluorescence imaging box; 1-6 weighing unit; 1-7 integrated water and fertilizer irrigation unit; 1-8 plant growth supplementary lighting unit; 1-9 central control unit; 1-10 display unit.

[0045] Reference Figure 2 , Figure 2 Figure 2 is a schematic diagram of the transmission structure of the pipeline provided by the present invention. Among them, it includes: 2-1 planting row; 2-2 transmission inlet row; 2-3 acquisition row; 2-4 fluorescence darkroom channel; 2-5 transmission outlet row.

[0046] The planting row is provided with multiple rows of roller structure conveyor lines arranged in parallel. The roller structure conveyor line carries the measured plants with identification signs. The transmission row is composed of a transmission inlet row, a transmission outlet row and a fluorescence darkroom channel. Among them, an inlet row detection unit is provided at the entrance of the transmission inlet row for identifying the identification sign of the measured plant. The planting row is connected to the transmission inlet row and the transmission outlet row through a steering structure. The transmission outlet row is connected to the fluorescence darkroom channel through a steering structure. The fluorescence darkroom channel is connected to the acquisition row. The acquisition row integrates a fluorescence imaging darkroom, a spectral imaging box and a 3D imaging box in the transmission direction to respectively perform plant fluorescence image acquisition, plant spectral image acquisition and plant multi-view image acquisition. The planting row, the transmission row and the acquisition row are respectively driven by independent drive motors. And limit detection mechanisms are provided at the connection between the transmission inlet row and the planting row, the connection between the planting row and the transmission outlet row, the connection between the transmission outlet row and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition row. The limit detection mechanism is used to monitor the transmission position of the measured plant. The drive motor and the limit detection mechanism are centrally controlled by the programmable logic controller of the central control unit. The central control unit communicates with the display unit to real-time feedback data. The display unit is used for the visualization operation interface, real-time monitoring of the equipment status, displaying the imaging data of the measured plant and the transmission position of the measured plant. Among them, the imaging data of the measured plant and the transmission position of the measured plant are matched with the identification sign.

[0047] The transmission structure of the pipeline is divided into a planting row, a transmission row and an acquisition row. Among them, the planting row is composed of multiple rows. The transmission row is divided into a transmission inlet row, a transmission outlet row and a fluorescence darkroom channel.

[0048] The plant 3D imaging box, the spectral imaging box, the fluorescence imaging box and the fluorescence darkroom channel are successively deployed on the acquisition row.

[0049] It should be noted that during the image acquisition process, it is necessary to sequentially perform plant fluorescence image acquisition, plant spectral image acquisition, and plant multi-view image acquisition (that is, the plant to be measured passes through the fluorescence darkroom channel, fluorescence imaging box, spectral imaging box, and plant 3D imaging box of the acquisition column in sequence through the conveyor belt). The plant to be measured needs to first pass through the darkroom to eliminate ambient light interference, and then enter the fluorescence, hyperspectral, and 3D imaging links in sequence.

[0050] Here, the fluorescence darkroom channel: provides an ambient light-free environment to avoid interference with fluorescence signals. The fluorescence imaging box: uses a specific wavelength light source to excite plant fluorescence and needs to be preferentially acquired after the darkroom. The spectral imaging box: acquires multi-band spectral data and needs to avoid light source contamination after fluorescence. The plant 3D imaging box: obtains a three-dimensional model through structured light / LiDAR and is finally executed to avoid movement interference.

[0051] Place a plant planting carrier on the planting column, place potted plants on the carrier, and hang a sample identification plate or QR code on one side of the carrier to identify the plant sample number.

[0052] At the entrance of the transfer-in column, an in-column detection unit is set up. The in-column detection unit consists of a position detection unit and a barcode scanner. The position detection unit is used to detect the entry of the plant sample into the column and pause, waiting for the barcode scanner (or QR code recognition unit) to identify the sample number and automatically enter the sample data.

[0053] Reference Figure 3 , Figure 3 is a schematic diagram of the distribution and connection of each unit of the pipeline platform provided by the present invention. Among them, it includes: a fluorescence box, a spectral box, a 3D box, a detection unit, a steering unit (i.e., a steering structure), a weighing unit, an irrigation unit, a roller conveyor structure, and a belt conveyor structure.

[0054] As Figure 3 shown, the transfer structures of each unit of the planting column, transfer column, and acquisition column are composed of a roller conveyor structure and a belt conveyor structure. Among them, the belt conveyor structure is located at both ends of the fluorescence darkroom channel, and the belt conveyor structure is also used for the weighing unit, and the rest are roller structures; a steering unit is installed at the connection of each unit to be used for steering during the plant transfer process; a driving motor is independently installed on each transfer line to control the uniform operation of the transfer unit.

[0055] At the joints of the planting column and the conveying-in column, the planting column and the conveying-out column, the conveying-out column and the fluorescence darkroom passage, the fluorescence darkroom passage and the fluorescence imaging darkroom, and at the central parts of the fluorescence imaging darkroom (including the fluorescence imaging box body), the spectral imaging room (including the spectral imaging box body), and the 3D imaging room (including the 3D imaging box body), limit detection mechanisms are installed to detect the conveying position of the flowerpots to be measured. The driving motor and the limit detection mechanism are electrically connected to the programmable logic controller in the central control room and are controlled by the central control unit.

[0056] Through the embodiments of the present invention, through the modular pipeline design of the planting column, the conveying column and the acquisition column, combined with the multi-column parallel roller conveying line, the independent driving motor and the steering structure, the full-process automatic transmission of the plants to be measured from planting to multi-modal data acquisition is realized; the acquisition column sequentially integrates the fluorescence imaging darkroom, the spectral imaging box body and the 3D imaging box body. Through the coordinated control of the limit detection mechanism and the central control unit, the requirements of different imaging conditions are accurately matched, and the high-throughput acquisition of morphological, spectral and physiological phenotype data is completed synchronously; at the same time, based on the identification plate recognition and real-time position tracking technology, the unique association between the data and the plants is ensured, and the device status and data flow are centrally monitored through the visualization interface, significantly improving the acquisition efficiency and data accuracy.

[0057] According to a plant multi-modal phenotype acquisition pipeline device provided by the present invention, the 3D imaging box body includes: Electric switch doors, which are respectively installed on the front side and the rear side of the 3D imaging box body; A rotating turntable, on the rotating arm of which a plurality of RGB image sensors are installed to form a multi-view image array sensor unit for collecting multi-view images of the plants to be measured; A lifting guide rail, on which at least one RGB image sensor is installed for collecting top-view images of the plants to be measured; Lighting lamps, which are respectively installed around the inner wall and on the top of the inner wall of the 3D imaging box body; Among them, the electric switch doors, the rotating turntable, the lifting guide rail and the lighting lamps are electrically connected to the central control unit through the 3D imaging controller. The central control unit respectively controls the opening and closing state of the electric switch doors, the rotation angle of the rotating turntable, the lifting height of the lifting guide rail and the brightness of the lighting lamps. The RGB image sensors are communicatively connected to the central control unit, and the central control unit is also used for collecting multi-view images and top-view images.

[0058] In the embodiments of the present invention, the imaging box body includes: the 3D imaging box body, the spectral imaging box body and the fluorescence imaging box body.

[0059] Among them, for the 3D imaging box body, electric switch doors are installed at the front and rear, a rotary turntable and a lifting guide rail are installed at the top, LED lighting lamps are installed at the top inner wall and around the inner wall, the electric switch doors, the rotary turntable, the lifting guide rail and the LED lighting lamps are electrically connected through a controller and a central control unit, and the central control unit acquisition system controls the switching, rotation and lifting; a plurality of RGB image sensors are installed on the rotary arm of the rotary turntable to form a multi-view image array sensor unit for collecting multi-view images of the side view of plants, and an RGB image sensor is installed on the lifting guide rail for collecting top-view images of plants; the RGB image sensors and the central control unit are network-connected, and the central control unit acquisition system collects multi-view RGB images and top-view images of plants.

[0060] Among them, for the spectral imaging box body, electric switch doors are installed at the front and rear, a lifting slide rail is installed at the top, a lifting guide rail is installed on the side wall, halogen spectral lamps are installed at the top inner wall and around the inner wall, the electric switch doors, the lifting slide rail and the halogen spectral lamps are electrically connected through a controller and a central control unit, and the central control unit acquisition system controls the switching and lifting; a spectral sensor is installed under the lifting slide rail at the top for collecting top-view spectral images of plants; a spectral sensor is installed on the side of the lifting guide rail on the side wall for collecting side-view spectral images of plants; among them, the spectral sensor is one or more of a hyperspectral sensor, a multispectral sensor, a thermal imaging sensor, etc.; the spectral sensor and the central control unit are network-connected, and the central control unit acquisition system controls the spectral camera to collect spectral images of plants.

[0061] Reference Figure 6 , Figure 6 is the internal phenotypic sensor distribution and mounting structure diagram of the imaging box body provided by the present invention. Among them, it includes: 6-1: the mounting structure diagram of the plant multi-view sensor array of the 3D imaging box body, 6-2 the mounting structure diagram of the plant spectral sensor of the spectral imaging box body, 6-3 the mounting structure diagram of the sensor array of the fluorescence imaging box body. It also includes: a rotating shaft, a guide rail slider, a sensor, a counterweight and a linear guide rail.

[0062] Through the embodiment of the present invention, the 3D imaging box body realizes automatic opening and closing through the front and rear electric switch doors, integrates a rotary turntable (with a multi-view RGB image array sensor) and a lifting guide rail (including a top-view RGB sensor) inside, and combines with a multi-angle LED lighting system. Under the centralized control of the central control unit, the rotation angle of the turntable, the height of the guide rail and the light intensity can be accurately adjusted, and multi-angle side-view and top-view high-resolution RGB images of plants can be obtained synchronously, realizing the all-round, automatic and high-precision acquisition of the three-dimensional morphological phenotypes of plants.

[0063] According to a plant multi-modal phenotypic acquisition pipeline device provided by the present invention, the fluorescence imaging darkroom includes: a fluorescence imaging box body and a dark treatment channel; Electric switch doors are installed on the front and rear sides of the fluorescence imaging box body. A lifting slide rail is installed on the top of the fluorescence imaging box body, and a fluorescence light source and a fluorescence camera are installed below the lifting slide rail. Among them, the electric switch door, the fluorescence light source, the fluorescence camera and the central control unit are electrically connected. The central control unit is also used to control the opening and closing state of the electric switch door, the on-off state of the fluorescence light source, and the fluorescence image acquisition of the fluorescence camera. The dark treatment channel is used to form a dark environment with the fluorescence imaging box body after the electric switch door is closed.

[0064] In the embodiment of the present invention, the fluorescence imaging darkroom includes: a fluorescence imaging box body and a dark treatment channel. The dark treatment channel is installed in front of the fluorescence imaging box body. Electric switch doors are installed on the front and rear of the fluorescence imaging box body. The fluorescence imaging box body and the dark treatment channel form a dark environment after the switch door is closed. A lifting slide rail is installed on the top, and a fluorescence light source and a fluorescence camera are installed below the lifting slide rail. The electric switch door, the fluorescence light source, the fluorescence camera and the central control unit are electrically connected. The central control unit of the acquisition system controls the opening and closing of the electric switch door, the on-off of the fluorescence light source, and the fluorescence image acquisition of the fluorescence camera.

[0065] Through the embodiment of the present invention, the fluorescence imaging darkroom forms a fully enclosed dark environment after the electric switch door is closed through the collaborative design of the dark treatment channel and the fluorescence imaging box body, effectively isolating external light interference and ensuring the high-precision darkroom conditions required for fluorescence imaging. The lifting slide rail on the top carries the fluorescence light source and the fluorescence camera. Combined with the centralized control of the central control unit, the height of the light source and the viewing angle of the camera can be dynamically adjusted to accurately adapt to the fluorescence excitation requirements of different plant heights and forms, realizing the automatic and standardized acquisition of plant fluorescence phenotype data.

[0066] According to a plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: A water and fertilizer integrated irrigation unit is located at one end of the incoming conveyor. The water and fertilizer integrated irrigation unit includes: a fertilizer tank, a fertilizer suction pump, an adjustable water outlet pipe, a flow meter and a solenoid valve; Among them, the flow meter and the solenoid valve are communicatively connected with the central control unit.

[0067] In the embodiment of the present invention, a water and fertilizer integrated irrigation unit is installed at one end of the incoming conveyor. The water and fertilizer integrated irrigation unit consists of a fertilizer tank, a fertilizer suction pump, an adjustable water outlet pipe, a flow meter and a solenoid valve. The flow meter and the solenoid valve are connected to the central control unit through electrical signals.

[0068] Through the embodiment of the present invention, quantitative water and fertilizer irrigation can be automatically completed when the plant enters the incoming conveyor. Through the programmed control of the irrigation time, flow rate and ratio by the central control unit, the consistency of the plant growth environment is ensured, the error of manual intervention is reduced, and at the same time, standardized pretreatment conditions are provided before phenotype acquisition.

[0069] According to a plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: A weighing unit, which is installed at one end of the acquisition column. The weighing unit includes a plurality of weighing sensors, and the weighing sensors are respectively located on the four corner support columns of the weighing conveyor line of the acquisition column; Among them, the weighing sensors are electrically connected to the central control unit.

[0070] In an embodiment of the present invention, a weighing unit is installed at one end of the acquisition column. The weighing unit consists of four weighing sensors installed on the four corner support columns of the weighing conveyor line, and the weighing sensors are connected to the central control unit through electrical signals.

[0071] According to a plant multi-modal phenotype acquisition pipeline device provided by the present invention, the device further includes: A plant growth supplementary lighting unit, which is suspended above the planting column, and a plurality of controllable and adjustable supplementary lighting units are installed according to the area size; The supplementary lighting unit is composed of a multi-node full-spectrum supplementary light, a supplementary light boom, a lifting system, and a supplementary light controller. The lifting system and the full-spectrum supplementary light are electrically connected to the supplementary light controller; The supplementary light controller is used to control the on / off of the full-spectrum supplementary light, the light quantum size of the full-spectrum supplementary light, and the spectral channel of the full-spectrum supplementary light. The supplementary light controller is also used to control the height of the supplementary light unit from the measured plants; The supplementary lighting unit covers the planting column in areas and is used to provide on-demand supplementary lighting for the measured plants on the planting column.

[0072] Reference Figure 4 , Figure 4 is a schematic structural diagram of the plant growth supplementary lighting unit provided by the present invention, which includes: 4-1 supplementary lighting unit 1; 4-2 supplementary lighting unit 2; 4-3 supplementary light boom; 4-4 full-spectrum supplementary light; 4-5 lifting system.

[0073] In an embodiment of the present invention, the plant growth supplementary lighting unit is suspended above the planting column, and a plurality of controllable and adjustable supplementary lighting units are installed according to the area. Each supplementary lighting unit is composed of a multi-node full-spectrum supplementary light, a supplementary light boom, a lifting system, and a supplementary light controller. The lifting system and the full-spectrum supplementary light are electrically connected to the supplementary light controller. The supplementary light controller controls the on / off of the full-spectrum supplementary light, the size of the light quantum, and the spectral channel, and controls the height of the supplementary light unit from the planted plants; the supplementary lighting unit covers the entire planting column in areas and provides on-demand supplementary lighting for the growth of the plants on the planting column.

[0074] Through the embodiments of the present invention, the plant growth light supplement unit realizes precise control of light parameters (photon intensity, spectral composition) and the height of the full-spectrum supplementary light lamps by arranging multi-node full-spectrum supplementary light lamps in different regions, in combination with a lifting system and a light supplement controller, and can dynamically adjust the light supplement mode according to the needs of different plants or growth stages; through regional coverage and independent control, it ensures uniform light in each region of the planting row and adapts to the growth characteristics of plants, avoiding insufficient or excessive local light.

[0075] Reference Figure 5 , Figure 5 is the connection structure diagram of the central control unit provided by the present invention.

[0076] The central control unit, as Figure 5 shown, the main modules of the central control unit include an industrial computer, a display, a switch, and a PLC controller.

[0077] The industrial computer is connected to the PLC controller through a switch, and receives and controls the motor node N, the position sensor N, the electric switch door N, the weighing sensor, the light source switch and dimmer N, the light supplement unit elevator N, and the water and fertilizer irrigation solenoid valve; Among them, the motors are composed of multiple ones. The motors installed on the production line include a belt drive power motor, a roller conveyor power motor, and a steering motor. The motors in the imaging box include: the turntable motor of the 3D imaging box, the lifting guide rail motor of the spectral imaging box, and the push-sweep motor of the fluorescence box.

[0078] Among them, the position sensors are installed at multiple points, including: the connection position between the planting row and the conveyor row, the connection position between the conveyor row and the collection row, the connection position between the dark treatment channel and the conveyor line, the connection position between the dark treatment channel and the fluorescence box conveyor line, the central induction positions of the 3D imaging box, the spectral imaging box, and the fluorescence imaging box, the water and fertilizer integration irrigation position, etc.

[0079] The electric switch doors include multiple switch control positions, including the front and rear switch doors of the 3D imaging box, the front and rear switch doors of the spectral imaging box, the front and rear switch doors of the fluorescence imaging box, and the switch door of the dark treatment channel.

[0080] The light source switches and dimmers are installed at different positions, including: a three-dimensional LED light source installed in the 3D imaging box body, a dimmer installed on each light source, connected to a PLC controller to control the switch and brightness of the light source; a three-dimensional halogen light source installed in the spectral imaging box body, a dimmer installed on each light source, connected to a PLC controller to control the switch and brightness of the light source; on the plant growth supplementary lighting unit, a dimmer is installed, connected to a PLC controller to control the switch and brightness of the light source as needed; the supplementary lighting unit elevator is connected to the PLC controller to control the height of the supplementary lighting unit according to the plant height and supplementary lighting requirements in different areas; the water and fertilizer irrigation solenoid valve is connected to the PLC controller to control the switch of the irrigation system to supply water and fertilizer for plant growth.

[0081] The industrial control computer is connected to the phenotypic sensors through a switch to control the acquisition and transmission of various sensors. The connected phenotypic sensors include different types of sensors such as image sensors, point cloud sensors, spectral sensors, thermal infrared sensors, and fluorescence sensors.

[0082] The industrial control computer is connected to the barcode scanner through a switch. When a (tested plant) sample enters the queue, it scans and identifies the sample number, records it in the sample experiment database, and locates the planting arrangement position number of the plant in the planting column.

[0083] A collection control system is deployed on the operating system of the industrial control computer to control the above modules to perform pipeline operation and high-throughput collection of phenotypic data.

[0084] The display unit is connected to the industrial control computer to display the operation interface of the collection control system. The operation interface of the collection control system is distributedly deployed, and distributed display units are installed on the 3D imaging box body, spectral imaging box body, and fluorescence imaging box body respectively. A display is connected to the PLC controller to display and interactively operate the PLC-related control.

[0085] According to a plant multi-modal phenotypic collection pipeline device provided by the present invention, the above device further includes: A calibration unit, including: RGB color calibration plates and a first image pixel registration plate on both sides of the center of the conveyor line in the 3D imaging box body. Among them, the RGB color calibration plate has a square plate surface, and a 24-color standard color card RGB image calibration plate pattern is sprayed on it; Spectral calibration plates and a second image pixel registration plate on both sides of the center of the conveyor line in the spectral imaging box body. Among them, the spectral calibration plate has a square plate surface, and a spectral calibration plate pattern with different reflectivities is sprayed on it; Fluorescence calibration plates and a third image pixel registration plate on both sides of the center of the conveyor line in the fluorescence imaging box body. Among them, the fluorescence calibration plate has a square plate surface, and a spectral calibration plate pattern with different reflectivities is sprayed on it; The first image pixel registration board, the second image pixel registration board, and the third image pixel registration board all have square plate surfaces, and are sprayed with an image pixel registration pattern composed of multiple black dots.

[0086] Reference Figure 7 , Figure 7 is the multi-modal phenotype data fusion calibration structure diagram provided by the present invention. Among them, it includes: 7-1: Acquisition column transfer line, 7-2: 3D imaging box sensor fusion calibration structure, 7-3: Spectral imaging box sensor fusion calibration structure, 7-4: Fluorescent imaging box sensor fusion calibration structure, 7-2-1: RGB color calibration board, 7-2-2: First image pixel registration board, 7-3-1: Spectral calibration board, 7-3-2: Second image pixel registration board, 7-4-1: Fluorescent calibration board, 7-4-2: Third image pixel registration board.

[0087] As Figure 7 shown, on both sides of the central part of each imaging box transfer line, different types of calibration boards are installed respectively, which are used for the registration and fusion of multi-modal phenotype data collected by the pipeline platform to form multi-modal integrated phenotype data.

[0088] Among them, an RGB color calibration board and a first image pixel registration board are installed on both sides of the center of the 3D imaging box transfer line. The RGB color calibration board is designed with a rectangular plate surface, and a 24-color standard color card RGB image calibration board pattern is sprayed above the plate surface. The first image pixel registration board is designed with a rectangular plate surface, and an image pixel registration pattern composed of multiple black dots is sprayed above the plate surface; Among them, a spectral calibration board and a second image pixel registration board are installed on both sides of the center of the spectral imaging box transfer line. The spectral calibration board is designed with a rectangular plate surface, and a spectral calibration board pattern with different reflectivities is sprayed above the plate surface. The second image pixel registration board is designed with a rectangular plate surface, and an image pixel registration pattern composed of multiple black dots is sprayed above the plate surface; Among them, a fluorescent calibration board and a third image pixel registration board are installed on both sides of the center of the fluorescent imaging box transfer line. The fluorescent calibration board is designed with a rectangular plate surface, and a spectral calibration board pattern with different reflectivities is sprayed above the plate surface. The third image pixel registration board is designed with a rectangular plate surface, and an image pixel registration pattern composed of multiple black dots is sprayed above the plate surface.

[0089] In the embodiment of the present invention, the data acquisition system has functions such as experimental management, plant sample entry and operation, fusion calibration data acquisition, operation control of different acquisition modes in pipeline operation, and working condition detection. The main technical links and operation modes are as follows: Before starting the pipeline phenotype acquisition experiment, an experimental file is established through the platform control acquisition system. The content of the experimental file includes: experiment name, experiment personnel, experiment date, experimental crop information, acquisition mode, data storage path, etc., and is managed informatization through a database.

[0090] Reference Figure 8 , Figure 8 is the schematic diagram of the arrangement rule of plant samples in the pipeline planting columns provided by the present invention.

[0091] The sample number code of (the plant to be measured) consists of 2 parts, denoted as N-M. N represents the column number where the sample is located, and M represents the row number where it is located. Through the coding of N-M, the arrangement position number of the sample in the pipeline planting column is uniquely determined, and this position number code is associated with the sample name one by one in the database.

[0092] In the embodiment of the present invention, an automatic sample column entry numbering operation is adopted to complete the positioning and placement of the experimental samples in the pipeline planting columns.

[0093] Reference Figure 9 , Figure 9 is the schematic diagram of the operation process of the sample column entry pipeline provided by the present invention.

[0094] The operation process of the sample column entry of the pipeline is as Figure 9 shown. A sample identification plate or two-dimensional code is hung on one side of the carrier to identify the plant sample number; the samples with identification codes are manually placed one by one at the entrance of the transfer column entry (for example, the sample column entry code scanning and detection point), the pipeline is run, and via the column entry detection unit, the pipeline is paused. The detection unit (barcode scanner or vision two-dimensional code recognizer) identifies the number of the sample, stores it in the sample number database, starts running the pipeline, and runs sequentially to the designated position until all samples are in the pipeline planting columns, completing the control operation of the sample column entry pipeline.

[0095] Reference Figure 10 , Figure 10 is the schematic diagram of the operation process of the sample water and fertilizer perfusion pipeline provided by the present invention.

[0096] Optionally, for the integrated water and fertilizer irrigation of plant samples, two irrigation modes are adopted. One is irrigation during the phenotype data acquisition process, and the other is to run the pipeline platform alone for water and fertilizer irrigation. The control operation process of the integrated water and fertilizer irrigation of the pipeline samples is as Figure 10 shown.

[0097] First, configure the water and fertilizer solution according to the experimental requirements; then set the irrigation amount for each sample node; automatically run the production line. After the in-place detection of the water and fertilizer integrated irrigation points, pause the production line, and automatically irrigate according to the irrigation amount of the corresponding sample node. Record the irrigation amount and irrigation time of the sample in the database. After the irrigation is completed, restart the operation of the production line until all the samples to be irrigated are completed and transferred to the original position, ending the control operation of the water and fertilizer integrated irrigation.

[0098] Before starting the automatic acquisition of plant phenotype data on the production line, pre-acquire the fusion calibration data in the box, and store the acquired fusion calibration data in the specified directory to provide calibration data for the later calibration of plant multi-modal data. Specifically, it includes: After manually adjusting the information such as the light source and sensor position of the 3D imaging box, drive the top-view RGB image sensor to continuously acquire multiple pieces of 3D imaging calibration plate data under the conditions that the box is airtight and no plants are placed. After manually adjusting the information such as the light source and sensor position of the spectral imaging box, drive the top-view spectral image sensor to continuously acquire multiple pieces of spectral imaging calibration plate data under the conditions that the box is airtight and no plants are placed. After manually adjusting the information such as the sensor position of the fluorescence imaging box, drive the top-view fluorescence image sensor to continuously acquire multiple pieces of fluorescence imaging calibration plate data under the conditions that the box is airtight and no plants are placed. Complete the acquisition of calibration data according to the above method.

[0099] In view of the acquisition condition requirements of different types of sensors and the user's usage requirements, set multiple automatic operation acquisition modes in the data acquisition system, including the following: Production line operation mode 1: In this mode, the acquisition of plant fluorescence phenotype data is not included, and it is divided into three acquisition modes: only acquiring data of the 3D imaging box, only acquiring data of the spectral imaging box, and synchronously acquiring data of the 3D imaging box and the spectral imaging box. According to Figure 10 The shown operation process drives the production line to run, specifically as follows: Respectively, in production line operation mode 1, in the mode of only acquiring data of the 3D imaging box, the production line is automatically conveyed in sequence, conveyed to the central part of the 3D imaging box, passes through the position detection unit, triggers the in-place signal, obtains the plant sample number, stops the operation of the production line, turns on the light source in the 3D imaging box, closes the front and rear switch doors of the 3D imaging box, automatically runs the turntable, turns on the sensors in the 3D imaging box to acquire data, and saves it to the specified directory; after the data acquisition is completed, open the front and rear switch doors of the 3D imaging box, stop the turntable operation, stop the sensor acquisition, and turn off the light source; continue to start the operation of the production line until all the measured plant samples complete the data acquisition.

[0100] Specifically, in the pipeline operation mode 1, in the mode of only collecting spectral imaging chamber data, the pipeline automatically conveys in sequence, conveys to the central part of the spectral imaging chamber, passes through the position detection unit, triggers the in-place signal, obtains the plant sample number, stops the pipeline operation, turns on the light source in the spectral imaging chamber, closes the front and rear switch doors of the spectral imaging chamber, the sensors in the spectral imaging chamber collect data, and saves it to the specified directory; after the data collection is completed, opens the front and rear switch doors of the spectral imaging chamber, stops the sensor collection, and turns off the light source; continues to start the pipeline operation until all the measured plant samples complete the data collection.

[0101] Specifically, in the pipeline operation mode 1, in the mode of synchronously collecting 3D imaging chamber data and spectral imaging chamber data, the pipeline automatically conveys in sequence.

[0102] First, it conveys to the central part of the spectral imaging chamber, passes through the position detection unit, triggers the in-place signal, obtains the plant sample number, stops the pipeline operation, turns on the light source in the spectral imaging chamber, closes the front and rear switch doors of the spectral imaging chamber, the sensors in the spectral imaging chamber collect data, and saves it to the specified directory; after the data collection is completed, opens the front and rear switch doors of the spectral imaging chamber, stops the sensor collection, turns off the light source, and continues to start the pipeline operation.

[0103] Next, it conveys to the central part of the 3D imaging chamber, passes through the position detection unit, triggers the in-place signal, obtains the plant sample number, stops the pipeline operation, turns on the light source in the 3D imaging chamber, closes the front and rear switch doors of the 3D imaging chamber, automatically operates the turntable, turns on the sensors in the 3D imaging chamber to collect data, and saves it to the specified directory; after the data collection is completed, opens the front and rear switch doors of the 3D imaging chamber, stops the turntable operation, stops the sensor collection, turns off the light source; synchronously, in the spectral imaging chamber, according to the spectral data collection process, completes the collection of spectral data. Continues to start the pipeline operation until all the measured plant samples complete the data collection.

[0104] Reference Figure 11 , Figure 11 is the schematic diagram of the operation process of the fluorescence data collection pipeline of the present invention.

[0105] Pipeline operation mode 2: In this mode, it includes collecting plant fluorescence phenotype data, which is divided into four collection modes: only collecting fluorescence data, synchronously collecting fluorescence data and spectral imaging chamber data, synchronously collecting fluorescence data and 3D imaging chamber data, and synchronously collecting fluorescence data, spectral imaging chamber data and 3D imaging chamber data. Drives the pipeline operation according to the Figure 11 shown operation process, specifically as follows: Separately, in the pipeline operation mode 2, in the mode of only collecting fluorescence data, first, the pipeline automatically transfers N plant samples to the dark treatment channel in sequence, stays for a specified time for the dark reaction of the plants. After the dark treatment ends, it continues to transfer the plant samples from the dark treatment channel to the fluorescence imaging box in sequence. At the central part, it is detected by the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the sensors in the fluorescence imaging box to collect data, and saving it to the specified directory; after the data collection is completed, open the front and rear switch doors of the fluorescence imaging box; continue to start the pipeline operation until all the measured plant samples complete the data collection.

[0106] Separately, in the pipeline operation mode 2, in the mode of synchronously collecting fluorescence data and spectral imaging box data, first, the pipeline automatically transfers N plant samples to the dark treatment channel in sequence, stays for a specified time for the dark reaction of the plants. After the dark reaction ends, it continues to transfer the plant samples from the dark treatment channel to the fluorescence imaging box in sequence. At the central part, it is detected by the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the sensors in the fluorescence imaging box to collect data, and saving it to the specified directory; after the data collection is completed, open the front and rear switch doors of the fluorescence imaging box; next, transfer it to the central part of the spectral imaging box, pass through the position detection unit, trigger the in-place signal, obtain the plant sample number, stop the pipeline operation, turn on the light source in the spectral imaging box, close the front and rear switch doors of the spectral imaging box, turn on the sensors in the spectral imaging box to collect data, and save it to the specified directory; after the data collection is completed, open the front and rear switch doors of the spectral imaging box, stop the sensor collection, and turn off the light source; synchronously, in the fluorescence imaging box, complete the collection of fluorescence data according to the fluorescence data collection process; until all the measured plant samples complete the data collection.

[0107] Specifically, in the pipeline operation mode 2, in the mode of synchronously collecting fluorescence data and 3D imaging chamber data, first, the pipeline automatically transfers N plant samples to the dark treatment channel in sequence, stays for a specified time for the dark reaction of the plants. After the dark reaction ends, the plant samples are continuously transferred from the dark treatment channel to the fluorescence imaging chamber in sequence. At the central part, they are detected by the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the sensors in the fluorescence imaging chamber to collect data, and saving it to the specified directory. After the data collection is completed, the front and rear switch doors of the fluorescence imaging chamber are opened. Next, the samples are transferred to the central part of the 3D imaging chamber, passed through the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the light source in the 3D imaging chamber, closing the front and rear switch doors of the 3D imaging chamber, automatically operating the turntable, turning on the sensors in the 3D imaging chamber to collect data, and saving it to the specified directory. After the data collection is completed, the front and rear switch doors of the 3D imaging chamber are opened, the turntable operation is stopped, the sensor collection is stopped, and the light source is turned off. Synchronously, in the fluorescence imaging chamber, the fluorescence data is collected according to the fluorescence data collection process until all the measured plant samples complete the data collection.

[0108] Specifically, in the pipeline operation mode 2, in the mode of synchronously collecting fluorescence data, spectral imaging chamber data and 3D imaging chamber data, first, the pipeline automatically transfers N plant samples to the dark treatment channel in sequence, stays for a specified time for the dark reaction of the plants. After the dark reaction ends, the plant samples are continuously transferred from the dark treatment channel to the fluorescence imaging chamber in sequence. At the central part, they are detected by the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the sensors in the fluorescence imaging chamber to collect data, and saving it to the specified directory. After the data collection is completed, the front and rear switch doors of the fluorescence imaging chamber are opened.

[0109] Next, the samples are transferred to the central part of the spectral imaging chamber, passed through the position detection unit, triggering the in-place signal, obtaining the plant sample number, stopping the pipeline operation, turning on the light source in the spectral imaging chamber, closing the front and rear switch doors of the spectral imaging chamber, turning on the sensors in the spectral imaging chamber to collect data, and saving it to the specified directory. After the data collection is completed, the front and rear switch doors of the spectral imaging chamber are opened, the sensor collection is stopped, and the light source is turned off. Synchronously, in the fluorescence imaging chamber, the fluorescence data is collected according to the fluorescence data collection process.

[0110] Next, it is transmitted to the central part of the 3D imaging box. After passing through the position detection unit, an in-place signal is triggered, the plant sample number is obtained, the assembly line operation is stopped, the light source inside the 3D imaging box is turned on, the front and rear switch doors of the 3D imaging box are closed, the turntable is automatically operated, the sensor inside the 3D imaging box is turned on to collect data, and the data is saved to the specified directory; after the data collection is completed, the front and rear switch doors of the 3D imaging box are opened, the turntable operation is stopped, the sensor collection is stopped, and the light source is turned off.

[0111] Synchronously, in the fluorescence imaging box, according to the fluorescence data collection process, the fluorescence data is collected, and in the spectral imaging box, according to the spectral data collection process, the spectral data is collected until all the measured plant samples complete the data collection.

[0112] Reference Figure 12 , Figure 12 is the flowchart of the multi-modal data fusion and phenotypic analysis algorithm provided by the present invention.

[0113] Taking the RGB image calibration plate image and the spectral image calibration plate image as inputs, an image color correction matrix and spectral correction parameters are respectively generated through the correction model.

[0114] The multi-view sequence images collected by the 3D imaging box, combined with the correction matrix of the RGB calibration plate and the SMF and MVS reconstruction algorithms, generate the three-dimensional point cloud of the scene corrected by RGB color.

[0115] The top-view spectral image collected by the spectral imaging box generates the corrected spectral image through the correction parameters of the spectral calibration plate, and provides the image pixel correction scale, translation matrix, and rotation matrix for geometric correction.

[0116] The top-view fluorescence image collected by the fluorescence imaging box is synchronously corrected.

[0117] Perform ratio correction on the three-dimensional point cloud of the scene based on the image pixel registration plate to generate the three-dimensional point cloud of the scene corrected by ratio.

[0118] Fuse the corrected spectral image, the three-dimensional point cloud of the scene corrected by RGB color, and the top-view fluorescence image to form a multi-modal data set (the three-dimensional point cloud and spectrum are fused and converted into the fusion of three-dimensional point cloud, spectrum, and fluorescence data of the scene).

[0119] Perform organ segmentation and plant segmentation through the deep learning module to extract the local and overall structure information of the plant. Finally, morphological phenotypes (such as three-dimensional structure), color phenotypes (RGB features), spectral phenotypes (spectral reflection characteristics), and fluorescence phenotypes are output.

[0120] For the same batch of experiments, the user adjusts the positions of the light sources and sensors in each imaging box according to the morphological and structural characteristics of the plants. Calibration data is collected. Among them, the RGB image calibration plate data collected by the top-view RGB image sensor in the 3D imaging box is denoted as: ; among them, the spectral calibration plate image data collected by the top-view spectral image sensor in the spectral imaging box is denoted as: ; among them, the fluorescence calibration plate image data collected by the top-view fluorescence image sensor in the fluorescence imaging box is denoted as: .

[0121] Through the RGB image , the median value of the pixels of 24 color patches is detected to form a camera RGB value matrix , and the standard reference value matrix . Using the linear model: , the image color correction matrix is calculated.

[0122] Through the spectral image , the standard calibration plate of the spectral reflectance is detected to construct a linear model: , is the reflectance of the specified band, and the spectral correction parameter is calculated. Among them, represents the true spectral reflectance of the target object, represents the measured original spectral radiation value (from the spectral sensor), represents the gain coefficient, represents the offset coefficient.

[0123] Through the RGB image , the image pixel registration plate is detected, the black markers distributed on it are recognized, the center points of two adjacent black markers are located, and the pixel distance between the two center points is calculated as L2, and its actual distance length is denoted as L1. The image pixel ratio correction parameter in the 3D imaging box is .

[0124] Through the spectral image , the image pixel registration plate is detected, the black markers distributed on it are recognized, the center points of two adjacent black markers are located, and the pixel distance between the two center points is calculated as L3, and its actual distance length is denoted as L1. The image pixel ratio correction parameter in the spectral imaging box is .

[0125] Through the fluorescence image , the image pixel registration board is detected, the black markers distributed thereon are recognized, the center points of two adjacent black markers are located, the pixel distance between the two center points is calculated as L4, and the actual distance length is denoted as L1. The image pixel ratio correction parameter in the fluorescence imaging box is .

[0126] Through the RGB image , the boundaries of the image pixel registration board and the color calibration board are detected. Taking the centers of the two calibration boards as the center points, the direction along the production line is the X-axis, and the vertical direction is the Y-axis. The translation matrix of the RGB image corrected to this coordinate system and the rotation matrix .

[0127] Through the spectral image , the boundaries of the image pixel registration board and the spectral calibration board are detected. Taking the centers of the two calibration boards as the center points, the direction along the production line is the X-axis, and the vertical direction is the Y-axis. The translation matrix of the spectral image corrected to this coordinate system and the rotation matrix .

[0128] Through the fluorescence image , the boundaries of the image pixel registration board and the fluorescence calibration board are detected. Taking the centers of the two calibration boards as the center points, the direction along the production line is the X-axis, and the vertical direction is the Y-axis. The translation matrix of the fluorescence image corrected to this coordinate system and the rotation matrix .

[0129] According to a plant multi-modal phenotype acquisition pipeline device provided by the present invention, the central control unit is further configured to: Obtain a multi-view image sequence of the measured plant collected by the 3D imaging box; Through a multi-view reconstruction algorithm, based on the multi-view image sequence, determine 3D point cloud data, where the 3D point cloud data includes: position points and vertex colors; Through the point cloud color detection algorithm, based on the position points and vertex colors of the 3D point cloud data and the preset image pixel registration board, perform point cloud restoration to obtain the reconstructed point cloud data, where the reconstructed point cloud data is the same size as the measured plant; Multiply the reconstructed point cloud data by the image color correction matrix generated based on the RGB color calibration board to obtain the corrected point cloud data, where the corrected point cloud data is in the same color space as the measured plant. For the same plant sample, the multi-view image sequence collected by the multi-view image array sensor unit in the 3D imaging box is denoted as .

[0130] Using the multi-view reconstruction algorithm SMF+MVS, with the plant multi-view image sequence as the input, the 3D point cloud data of the measured scene is reconstructed, including position points and vertex color information, denoted as .

[0131] Based on the point cloud color detection algorithm, in , for the bottom point cloud, a series of black dots are detected. The average diameter d1 of the dots in the point cloud is calculated and compared with the actual diameter d2 of the dots in the image pixel registration plate. Let t = d2 / d1. Through the formula: , the restored point cloud is obtained, and the reconstructed point cloud is restored to the same size as the actual object at a ratio of 1:1.

[0132] The RGB color channels of each vertex of the point cloud are multiplied by the image color correction matrix , and the restored and corrected point cloud data of the reconstructed scene is obtained. This point cloud data has the same state as the actual scene in terms of size and color space.

[0133] Through the above algorithm steps, the true 3D reconstruction of the measured plant in the 3D imaging box is realized. The reconstructed point cloud has 6-channel information. [[ID=(30)]]

[0134] Through multi-view image reconstruction, point cloud color correction and spatial registration, the generation of a high-fidelity three-dimensional model of the measured plant is realized, which can restore the geometric shape and true color of the plant, ensure that the three-dimensional point cloud is consistent with the physical object in terms of size ratio and color space, and provide a high-precision digital basis for plant phenotype analysis.

[0135] For the same plant sample, a single spectral image of the plant collected by the top-view spectral image sensor in the spectral imaging box is denoted as . Through the proportional correction parameter , the proportionally corrected image is denoted as: . Through the translation matrix and the rotation matrix , the spectral image is corrected to the same position as the RGB image , denoted as . Through the alignment of the top view and the point cloud projection, the spectral channel information of the corresponding points is obtained, and the three-dimensional fusion of the spectral data on the point cloud is realized. The obtained multi-channel spectral point cloud is denoted as , including vertex coordinate information , and n spectral channel information .

[0136] For the same plant sample, a single fluorescence image of the plant collected by the top-view fluorescence image sensor in the fluorescence imaging box is denoted as . Through the proportional correction parameter , the image after proportional correction is denoted as: . Through the translation matrix and the rotation matrix , the fluorescence image is corrected to the same position as the RGB image , denoted as .

[0137] By aligning the top view and the point cloud projection, the fluorescence channel information of the corresponding points is obtained, and the three-dimensional fusion of fluorescence data on the point cloud is realized, obtaining the multi-channel spectrum and fluorescence multi-modal point cloud (scene multi-modal point cloud): , including vertex coordinate information , n spectral channel information , k fluorescence channel information .

[0138] Through the above method, the three-dimensional fusion registration of pipeline multi-modal phenotype data is realized.

[0139] Taking the reconstructed and fused scene multi-modal point cloud as the input, the pipeline phenotype analysis is carried out, and the specific process is as follows: First, using the plant extraction deep learning model, the multi-modal point cloud data of the plant is extracted from the scene multi-modal point cloud , denoted as: ; Then, for different plant type characteristics, a deep learning semantic model is constructed and trained. Taking the multi-modal point cloud data of the plant as the input, organ segmentation is carried out, and the organ point clouds of plant leaves, stems, flowers, fruits, etc. are segmented, denoted as: , , , ; Taking the multi-modal point cloud data of the plant as the input, using methods such as point cloud measurement, color statistics, spectral inversion, and fluorescence calculation, the multi-modal phenotype indicators of the plant are analyzed from the plant type scale. The multi-modal phenotype indicators include morphological structure, color texture, physiological and biochemical, chlorophyll fluorescence, etc.; Taking the segmented organ point clouds of plant leaves, stems, flowers, fruits, etc., , , , Taking [the input], by using methods such as point cloud measurement, color statistics, spectral inversion, and fluorescence calculation, plant multi-modal phenotypic indicators are analytically obtained from the organ scale. The multi-modal phenotypic indicators include morphological structure, color texture, physiological and biochemical characteristics, chlorophyll fluorescence, etc.; Through the above methods, the pipelined analysis of plant multi-modal three-dimensional phenotypes is completed.

[0140] Through the above embodiments of the present invention, a plant three-dimensional multi-modal phenotype acquisition pipeline platform and a fusion analysis method are constructed. Among them, it includes: building a phenotypic acquisition device, an imaging box body, and a fusion calibration unit in the form of an automated conveyor belt for high-throughput automated acquisition of plant phenotypic data from different types of sensors; constructing an automated phenotypic acquisition system that can collect in an orderly and collaborative manner fully automatically without manual intervention according to the operation process; constructing a plant three-dimensional reconstruction and multi-modal phenotype fusion analysis method, based on the multi-modal image data obtained from the acquisition platform, realizing the three-dimensional modal point cloud reconstruction of plants, and the automated analysis of phenotypic indicators such as morphological structure, color texture, physiological and biochemical characteristics, and chlorophyll fluorescence at the plant scale and organ scale.

[0141] Effectively solve the problems of high-throughput acquisition and analysis of plant automated phenotypic traits. The effects of the present invention are mainly reflected in the following aspects: pipeline-style fully automated acquisition, which can carry out automated acquisition of crop phenotypes and continuous detection of the growth process. The design of the high-throughput imaging box body can synchronously collect phenotypic data of different sensor types. An automated acquisition, automated fusion, and analysis technology method for phenotypic data are constructed.

[0142] In an example of the embodiment of the present invention, taking the phenotype acquisition of the lettuce variety in the reproductive growth stage grown in a greenhouse as an example, the general plant height of lettuce in the reproductive growth stage is less than 1.0 m, and the width is less than 0.3 m. There are a total of 300 lettuce samples, which are planted in pots in a greenhouse. The height of the potted pot is 20 cm, and the matrix is covered to the edge of the pot. Based on the device and method provided by the present invention, the data acquisition process and data processing process are described.

[0143] Among them, for the planting rows, 20 samples are planted in each row, and a total of 15 planting rows are set.

[0144] Size of the 3D imaging box body: length 2.5 m width 2.5 m height 2.5 m; Size of the spectral imaging box body: length 2 m width 2 m height 2.5 m; Size of the fluorescence imaging box body: length 2 m width 2 m height 2.5 m; Size of the dark treatment channel: length 16 m width 2 m 2.5 m in height; Based on the plant multi-modal three-dimensional phenotype analysis method, the multi-modal phenotype indicators related to plant type obtained by analysis include morphological structure (plant height, projected area, total leaf area, compactness, volume, etc.), color texture (color mean value in the RGB channel), physiological and biochemical (inverted chlorophyll content, N content, disease and other indicators), chlorophyll fluorescence (minimum fluorescence F0, maximum fluorescence Fm, variable fluorescence Fv and other indicators), etc.

[0145] The plant multi-modal phenotype acquisition pipeline method provided by the present invention will be described below. The plant multi-modal phenotype acquisition pipeline method described below can be mutually corresponding and referred to the plant multi-modal phenotype acquisition pipeline device described above.

[0146] The measured plants with identification plates are conveyed through a roller structure conveyor line arranged in multiple parallel rows in the planting row; When the measured plants enter the incoming row of the conveying row, the identification plates are recognized by the incoming row detection unit; The measured plants are sequentially turned from the planting row to the incoming row, outgoing row and fluorescence darkroom channel through the turning structure, and finally conveyed to the acquisition row; The measured plants are sequentially subjected to plant fluorescence image acquisition, plant spectral image acquisition and plant multi-view image acquisition in the acquisition row according to the conveying direction; Among them, the planting row, the conveying row and the acquisition row are respectively driven by independent driving motors; at the connection between the planting row and the incoming row of the conveying row, the connection between the planting row and the outgoing row of the conveying row, the connection between the outgoing row of the conveying row and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition row, the conveying position of the measured plants is monitored in real time through the limit detection mechanism; The driving motors and the limit detection mechanism are centrally controlled by the programmable logic controller of the central control unit, and the real-time data is fed back to the display unit. Through the visualization operation interface of the display unit, the device status is monitored in real time, and the imaging data of the measured plants and the conveying position of the measured plants are displayed, wherein the imaging data of the measured plants and the conveying position of the measured plants are matched with the identification plates.

[0147] Specifically, the above-mentioned plant multi-modal phenotype acquisition pipeline device provided by the present invention can implement all the method steps implemented by the above-mentioned plant multi-modal phenotype acquisition pipeline method embodiment, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiment will not be specifically described herein.

[0148] Figure 13 It is a schematic physical structure diagram of the electronic device provided by the present invention, as Figure 13As shown in the figure, the electronic device may include: a processor 1310, a communications interface 1320, a memory 1330, and a communication bus 1340. Among them, the processor 1310, the communications interface 1320, and the memory 1330 complete communication with each other through the communication bus 1340. The processor 1310 may call the logic instructions in the memory 1330 to execute the plant multi-modal phenotype acquisition pipeline method, which includes: conveying the measured plant with an identification plate through a roller structure conveyor line arranged in multiple parallel columns of the planting column; when the measured plant enters the transfer-in column of the transfer column, using the in-column detection unit to identify the identification plate; turning the measured plant from the planting column to the transfer-in column, the transfer-out column, and the fluorescence darkroom channel in sequence through a turning structure, and finally conveying it to the acquisition column; sequentially performing plant multi-view image acquisition, plant spectral image acquisition, and plant fluorescence image acquisition on the measured plant in the acquisition column in the transfer direction; where the planting column, the transfer column, and the acquisition column are respectively driven by independent drive motors; at the connection between the planting column and the transfer-in column, the connection between the planting column and the transfer-out column, the connection between the transfer-out column and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition column, the transfer position of the measured plant is monitored in real time through a limit detection mechanism; the drive motors and the limit detection mechanism are centrally controlled through the programmable logic controller of the central control unit, and the real-time data is fed back to the display unit. Through the visualization operation interface of the display unit, the device status is monitored in real time, and the imaging data of the measured plant and the transfer position of the measured plant are displayed, where the imaging data of the measured plant and the transfer position of the measured plant match the identification plate.

[0149] In addition, when the logic instructions in the above-mentioned memory 1330 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0150] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the plant multi-modal phenotype acquisition pipeline method provided by the above-mentioned various methods. The method includes: conveying a measured plant with an identification plate through a roller structure conveyor line arranged in multiple parallel columns in the planting column; when the measured plant enters the incoming column of the conveying column, using the incoming column detection unit to identify the identification plate; turning the measured plant from the planting column to the incoming column, the outgoing column and the fluorescence darkroom channel in sequence through a turning structure, and finally conveying it to the acquisition column; sequentially performing plant fluorescence image acquisition, plant spectral image acquisition and plant multi-view image acquisition on the measured plant in the acquisition column according to the conveying direction; wherein, the planting column, the conveying column and the acquisition column are respectively driven by independent driving motors; at the connection between the planting column and the incoming column, the connection between the planting column and the outgoing column, the connection between the outgoing column and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition column, the conveying position of the measured plant is monitored in real time through a limit detection mechanism; the driving motors and the limit detection mechanism are centrally controlled through the programmable logic controller of the central control unit, and the real-time data is fed back to the display unit. Through the display unit, a visual operation interface, real-time monitoring of the device status, display of the imaging data of the measured plant and the conveying position of the measured plant are realized, wherein the imaging data of the measured plant and the conveying position of the measured plant are matched with the identification plate.

[0151] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it realizes the plant multi-modal phenotype acquisition pipeline method provided by the above-mentioned various methods. The method includes: conveying a measured plant with an identification plate through a roller structure conveyor line arranged in multiple parallel columns in the planting column; when the measured plant enters the incoming column of the conveying column, using the incoming column detection unit to identify the identification plate; turning the measured plant from the planting column to the incoming column, the outgoing column and the fluorescence darkroom channel in sequence through a turning structure, and finally conveying it to the acquisition column; sequentially performing plant fluorescence image acquisition, plant spectral image acquisition and plant multi-view image acquisition on the measured plant in the acquisition column according to the conveying direction; wherein, the planting column, the conveying column and the acquisition column are respectively driven by independent driving motors; at the connection between the planting column and the incoming column, the connection between the planting column and the outgoing column, the connection between the outgoing column and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition column, the conveying position of the measured plant is monitored in real time through a limit detection mechanism; the driving motors and the limit detection mechanism are centrally controlled through the programmable logic controller of the central control unit, and the real-time data is fed back to the display unit. Through the display unit, a visual operation interface, real-time monitoring of the device status, display of the imaging data of the measured plant and the conveying position of the measured plant are realized, wherein the imaging data of the measured plant and the conveying position of the measured plant are matched with the identification plate.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0153] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A plant multi-modal phenotype acquisition pipeline device, characterized in that, Including: A planting column, a conveying column, and a collection column, where: The planting column is provided with a plurality of roller structure conveyor lines arranged in parallel, and the roller structure conveyor line carries the measured plants with identification plates; The conveying column is composed of a conveying-in column, a conveying-out column, and a fluorescence darkroom channel. Wherein, an in-column detection unit is provided at the entrance of the conveying-in column for identifying the identification plate of the measured plant; The planting column is connected to the conveying-in column and the conveying-out column through a steering structure, and the conveying-out column is connected to the fluorescence darkroom channel through a steering structure; The fluorescence darkroom channel is connected to the collection column, and the collection column is sequentially integrated with a fluorescence imaging darkroom, a spectral imaging box body, and a 3D imaging box body in the conveying direction to respectively perform plant fluorescence image acquisition, plant spectral image acquisition, and plant multi-view image acquisition; The planting column, the conveying column, and the collection column are respectively driven by independent driving motors, and limit detection mechanisms are provided at the connection between the conveying-in column and the planting column, the connection between the planting column and the conveying-out column, the connection between the conveying-out column and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the fluorescence imaging darkroom. The limit detection mechanism is used to monitor the conveying position of the measured plant; The driving motor and the limit detection mechanism are centrally controlled by a programmable logic controller of the central control unit. The central control unit communicates with the display unit to feedback data in real time. The display unit is used for a visual operation interface, real-time monitoring of the equipment status, displaying the imaging data of the measured plant and the conveying position of the measured plant. Wherein, the imaging data of the measured plant and the conveying position of the measured plant are matched with the identification plate.

2. The plant multi-modal phenotype acquisition pipeline device according to claim 1, wherein The 3D imaging box body includes: Electric switch doors, which are respectively installed on the front side and the rear side of the 3D imaging box body; A rotating turntable, on the rotating arm of the rotating turntable, a plurality of RGB image sensors are installed to form a multi-view image array sensor unit for collecting multi-view images of the measured plant; A lifting guide rail, on which at least one RGB image sensor is installed for collecting top-view images of the measured plant; Lighting lamps, which are respectively installed around the inner wall and on the top of the inner wall of the 3D imaging box body; Wherein, the electric switch doors, the rotating turntable, the lifting guide rail, and the lighting lamps are electrically connected to the central control unit through a 3D imaging controller. The central control unit respectively controls the opening and closing state of the electric switch doors, the rotation angle of the rotating turntable, the lifting height of the lifting guide rail, and the brightness of the lighting lamps. The RGB image sensors are communicatively connected to the central control unit, and the central control unit is also used for collecting the multi-view images and the top-view images.

3. The plant multi-modal phenotype acquisition pipeline device according to claim 1, characterized in that The fluorescence imaging darkroom includes: a fluorescence imaging box body and a dark treatment channel; Electric switch doors are installed on the front side and the rear side of the fluorescence imaging box body, a lifting slide rail is installed on the top of the fluorescence imaging box body, and a fluorescence light source and a fluorescence camera are installed below the lifting slide rail; Among them, the electric switch door, the fluorescent light source, the fluorescent camera are electrically connected to the central control unit, and the central control unit is further configured to control the opening and closing state of the electric switch door, the on-off state of the fluorescent light source, and the acquisition of fluorescent images by the fluorescent camera; The dark treatment channel is used to form a dark environment with the fluorescence imaging box body after the electric switch door is closed.

4. The plant multi-modal phenotype acquisition pipeline device according to claim 1, characterized in that, The device further includes: A water and fertilizer integrated irrigation unit, which is located at one end of the incoming conveyor line. The water and fertilizer integrated irrigation unit includes: a fertilizer tank, a fertilizer suction pump, an adjustable water outlet pipe, a flow meter and a solenoid valve; Among them, the flow meter and the solenoid valve are communicatively connected to the central control unit.

5. The plant multi-modal phenotype acquisition pipeline device according to claim 1, characterized in that The device further includes: A weighing unit, which is installed at one end of the collection column. The weighing unit includes a plurality of weighing sensors, and the weighing sensors are respectively located on the four corner support columns of the weighing conveyor line of the collection column; Among them, the weighing sensors are electrically connected to the central control unit.

6. The plant multi-modal phenotype acquisition pipeline device according to claim 1, characterized in that, The device further includes: A plant growth supplementary lighting unit, which is suspended above the planting column, and a plurality of controllable and adjustable supplementary lighting units are installed according to the area size; The supplementary lighting unit is composed of a multi-node full-spectrum supplementary light, a supplementary light boom, a lifting system, and a supplementary light controller, and the lifting system and the full-spectrum supplementary light are electrically connected to the supplementary light controller; The supplementary light controller is used to control the on-off of the full-spectrum supplementary light, the light quantum size of the full-spectrum supplementary light, the spectral channel of the full-spectrum supplementary light, and the supplementary light controller is also used to control the height of the supplementary light unit from the measured plant; The supplementary lighting unit covers the planting column in a divided area, and is used to supplement light to the measured plants on the planting column as needed.

7. The plant multi-modal phenotype acquisition pipeline device according to claim 1, characterized in that The device further includes: A calibration unit, including: An RGB color calibration board and a first image pixel registration board located on both sides of the center of the conveyor line of the 3D imaging box body. Among them, the RGB color calibration board is a square board surface, and is sprayed with a 24-color standard color card RGB image calibration board pattern; A spectral calibration board and a second image pixel registration board located on both sides of the center of the conveyor line of the spectral imaging box body. Among them, the spectral calibration board is a square board surface, and is sprayed with a spectral calibration board pattern with different reflectivities; A fluorescent calibration board and a third image pixel registration board located on both sides of the center of the conveyor line of the fluorescence imaging box body. Among them, the fluorescent calibration board is a square board surface, and is sprayed with a spectral calibration board pattern with different reflectivities; The first image pixel registration board, the second image pixel registration board, and the third image pixel registration board are all square board surfaces, and are sprayed with an image pixel registration pattern composed of a plurality of black dots.

8. The plant multi-modal phenotype acquisition pipeline device according to claim 7, characterized in that The central control unit is further configured to: Obtain a multi-view image sequence of the measured plant collected by the 3D imaging box body; Through a multi-view reconstruction algorithm, based on the multi-view image sequence, determine 3D point cloud data, where the 3D point cloud data includes: position points and vertex colors; Through the point cloud color detection algorithm, based on the position points and vertex colors of the 3D point cloud data and a preset image pixel registration board, the point cloud is restored to obtain the reconstructed point cloud data, where the scale of the reconstructed point cloud data is the same as that of the measured plant. Multiply the reconstructed point cloud data by the image color correction matrix generated based on the RGB color calibration board to obtain the corrected point cloud data, where the color space of the corrected point cloud data is the same as that of the measured plant.

9. A method for a plant multi-modal phenotype acquisition pipeline, characterized in that, It includes: Convey the measured plant with an identification sign through a roller structure conveyor line arranged in multiple parallel rows of the planting row. When the measured plant enters the conveying inlet row of the conveying row, use the inlet row detection unit to identify the identification sign. Through the steering structure, the measured plant is sequentially steered from the planting row to the conveying inlet row, the conveying outlet row, and the fluorescence darkroom channel, and finally conveyed to the acquisition row. Through the acquisition row, sequentially collect plant fluorescence images, plant spectral images, and plant multi-view images of the measured plant in the conveying direction. Among them, the planting row, the conveying row, and the acquisition row are respectively driven by independent drive motors; at the connection between the planting row and the conveying inlet row, the connection between the planting row and the conveying outlet row, the connection between the conveying outlet row and the fluorescence darkroom channel, and the connection between the fluorescence darkroom channel and the acquisition row, the conveying position of the measured plant is monitored in real time through a limit detection mechanism. The programmable logic controller of the central control unit centrally controls the drive motor and the limit detection mechanism, and feeds the real-time data back to the display unit. Through the visualization operation interface of the display unit, the device status is monitored in real time, and the imaging data of the measured plant and the conveying position of the measured plant are displayed, where the imaging data of the measured plant and the conveying position of the measured plant are matched with the identification sign.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the plant multi-modal phenotype acquisition pipeline method as described in claim 9.

Images