Mobile portal frame phenotype acquisition device

Abstract

The utility model provides a mobile portal frame phenotype acquisition device. The device can be used for collecting plant phenotype data. The device is provided with the transverse track and the longitudinal track at the same time, so that the device can move conveniently, and phenotypic data can be collected from one area to another area. In the device, the image acquisition device connected to the manual holder can comprise a camera and lighting equipment, and can be adjusted according to the requirement of image acquisition. In addition, in the device, the detection device is detachable and convenient to use.

Description

Technical Field

[0001] The utility model belongs to the field of agricultural robots, and more specifically, relates to a mobile gantry phenotype collection device. Background Art

[0002] At present, the world's major breeding companies and major research institutions have realized that molecular design breeding is the key to significantly improving food yield, quality, resistance and other traits in the future. The core of molecular design breeding is to use modern biotechnologies such as current genome sequencing, whole genome association analysis and genome editing to make full use of the natural variation of crops to design and create excellent varieties with specific traits. However, to effectively use these biotechnologies, the current "bottleneck" problem is to obtain phenotypes for different traits of crops under different environmental conditions on a large scale, and then combine genome sequence information with modern molecular genetic methods to clarify the genetic variation that controls key traits.

[0003] Therefore, this field urgently needs to establish a platform with powerful phenotypic and genomic data collection capabilities, and to establish matching data analysis and mining tools, as well as new crop design and transformation facilities. Utility Model Content

[0004] The purpose of the utility model is to provide a mobile gantry phenotypic acquisition device. The device can be used to collect plant phenotypic data. Since the device has both horizontal and vertical tracks, it can be easily moved and expanded from one area to another to collect phenotypic data. In the device, the image acquisition device connected to the manual pan-tilt head can include a camera and a lighting device, and can be adjusted according to the needs of image acquisition. In addition, in the device, the detection device is detachable and easy to use.

[0005] In a first aspect of the utility model, a mobile gantry phenotype collection device is provided, the device comprising: a track portion consisting of a track beam, embedded parts under the track beam, and a gantry containing a detection device;

[0006] The track section includes: a horizontal track and a vertical track;

[0007] The gantry includes: an end beam, a main beam track, a lifting wheel group, a lifting wheel, a moving wheel, a platform car running mechanism and a trolley running mechanism; wherein the main beam track is parallel to the ground, a detection device is detachably and movably connected to the main beam track, and the detection device includes a trolley running mechanism for controlling the movement of the detection device on the main beam track; the end beam and the main beam track are connected together; the moving wheel can change direction to achieve switching between the horizontal track and the vertical track; the platform car running mechanism is connected to one side of the end beam to control the movement of the gantry on the vertical track; the trolley running mechanism is connected to the other side of the end beam to control the movement of the gantry on the horizontal track.

[0008] In one or more embodiments, the lifting wheel group is connected to the lower end of the end beam and is used to adjust the height of the main beam track from the ground; wherein the lifting wheel group is composed of a plurality of lifting wheels.

[0009] In one or more embodiments, the detection device further includes an adapter plate, and the detection device is fixed and disassembled by using a slot on the adapter plate.

[0010] In one or more embodiments, the track beam is composed of a track beam body, and a fixed component and a slide rail located on the track beam body.

[0011] In one or more embodiments, the embedded part is composed of an embedded supporting component, a connecting component and an embedded bracket, wherein the connecting component connects the embedded supporting component and the embedded bracket together.

[0012] In one or more implementation schemes, there are four pre-embedded brackets, which are respectively connected to four connecting parts of the pre-embedded supporting part.

[0013] In one or more embodiments, the detection device includes a trolley traveling mechanism, a manual pan-tilt head connected to the trolley traveling mechanism, and an image acquisition device connected to the manual pan-tilt head; wherein the image acquisition device is detachably connected to the manual pan-tilt head.

[0014] In one or more embodiments, the manual pan-tilt head includes a sphere, a pan-tilt head main knob, a panoramic base knob, a 360° rotating base scale, a clamp base knob, a clamp seat and a sleeve; the clamp base knob and the clamp seat cooperate with each other to adjust the tightness of the clamp seat, and are connected to the trolley walking mechanism; the sphere connects the clamp seat and the sleeve together, and cooperates with the panoramic base knob and the 360° rotating base scale to rotate the sleeve; the pan-tilt head main knob is used to rotate the manual pan-tilt head.

[0015] In one or more embodiments, the image acquisition device includes a camera and an optional lighting device.

[0016] In one or more embodiments, the image acquisition device further includes a heat shield.

[0017] Other aspects of the present invention will be apparent to those skilled in the art from the disclosure herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 , structural drawings of track beams and embedded parts.

[0019] Figure 2 , schematic diagram of mobile gantry.

[0020] Figure 3 , schematic diagram of the mobile gantry phenotype collection device.

[0021] Figure 4 , schematic diagram of manual gimbal.

[0022] Figure 5 , collection flow chart.

[0023] The corresponding reference numerals are as follows:

[0024] 1. Track beam;

[0025] 101. Track beam body;

[0026] 102. Fixed parts;

[0027] 103. Slide rail;

[0028] 2. Embedded parts;

[0029] 201. Embedded supporting components;

[0030] 202, connecting parts;

[0031] 203. Embedded bracket;

[0032] 3. Track;

[0033] 301, vertical (Y) track;

[0034] 302, horizontal (X) track;

[0035] 4. Gantry;

[0036] 401, end beam;

[0037] 402, main beam (Z) track;

[0038] 403, lifting wheel set;

[0039] 404, lifting wheel;

[0040] 405, moving wheel;

[0041] 406. Platform vehicle travel mechanism;

[0042] 407. Trolley travel mechanism;

[0043] 5. Detection device;

[0044] 501. Trolley travel mechanism;

[0045] 502, manual pan / tilt;

[0046] 503. Image acquisition device;

[0047] 601. Sphere;

[0048] 602, PTZ main knob;

[0049] 603, Panoramic base knob;

[0050] 604, 360° rotating base scale;

[0051] 605, clamp base knob;

[0052] 606, clamp seat;

[0053] 607. Sleeve. DETAILED DESCRIPTION

[0054] The utility model provides a mobile gantry phenotype collection device, which comprises: a track part composed of a track beam and embedded parts under the track beam, and a gantry containing a detection device.

[0055] In one or more embodiments, the track portion is composed of a track beam and embedded parts under the track beam, and is the basic part for the movement of the mobile gantry.

[0056] In one or more embodiments, the track beam is composed of a track beam body, and a fixed component and a slide rail located on the track beam body.

[0057] In one or more embodiments, the embedded part consists of an embedded supporting component, a connecting component and an embedded bracket.

[0058] In one or more embodiments, the embedded bracket is embedded in the concrete foundation.

[0059] In one or more embodiments, the embedded bracket is in a hook shape to increase the stability of the embedded part.

[0060] In one or more embodiments, the number of pre-embedded stents is 2, 3, 4, 5 or more.

[0061] In one or more implementation schemes, there are four pre-embedded brackets, which are respectively connected to the four end points of the pre-embedded supporting component.

[0062] In one or more embodiments, the connecting component connects the embedded supporting component and the embedded bracket together.

[0063] In one or more embodiments, embedded support members are used to support the track beams.

[0064] In one or more embodiments, the track portion includes: a horizontal (X) track and a vertical (Y) track.

[0065] In one or more embodiments, the track portion consists of a horizontal (X) track and a vertical (Y) track.

[0066] In one or more embodiments, the moving wheels of the mobile gantry can change direction to achieve switching between horizontal (X) tracks and vertical (Y) tracks.

[0067] In one or more embodiments, the area of ​​the track portion generally coincides with the area to be inspected.

[0068] In one or more embodiments, the gantry spans the same width as the vertical (Y) track of the track section.

[0069] In one or more embodiments, the area of ​​the track portion is 20m wide (Y track) and 80m long (X track).

[0070] In one or more embodiments, the gantry includes: an end beam, a main beam (Z) track, a lifting wheel group, a lifting wheel, a moving wheel, a platform trolley traveling mechanism and a cart traveling mechanism; wherein the end beam and the main beam (Z) track are connected together, the platform trolley traveling mechanism is connected to one side of the end beam to control the movement of the gantry on the vertical (Y) track, and the cart traveling mechanism is connected to the other side of the end beam to control the movement of the gantry on the horizontal (X) track.

[0071] In one or more embodiments, the trolley travel mechanism includes a drag chain electric tractor for towing the entire gantry to move on a horizontal (X) track.

[0072] In one or more embodiments, the platform vehicle walking mechanism is a rack platform vehicle, which drives the gantry to move on a vertical (Y) track through a chain. In one or more embodiments, the end beam plays a supporting role, usually in a triangular shape, for supporting the entire gantry and maintaining the stability of the gantry.

[0073] In one or more embodiments, the main beam (Z) track is parallel to the ground.

[0074] In one or more embodiments, the maximum load capacity of the main beam (Z) track is 0.2t.

[0075] In one or more embodiments, the main beam (Z) track is detachably connected with a detection device.

[0076] In one or more embodiments, the detection device includes a trolley travel mechanism for controlling the movement of the detection device on the main beam (Z) track.

[0077] In one or more embodiments, the trolley travel mechanism of the detection device is detachably and movably connected to the main beam (Z) track.

[0078] In one or more embodiments, a lifting wheel set is connected to the lower end of the end beam and is used to adjust the height of the main beam track from the ground.

[0079] In one or more embodiments, the lifting wheel assembly consists of a plurality of lifting wheels.

[0080] In one or more embodiments, the lifting wheel assembly can lift the gantry to a height of 5 m.

[0081] In one or more embodiments, the maximum load capacity of the gantry is 0.2 t.

[0082] In one or more embodiments, the lower end of the lifting wheel group is connected to a moving wheel, which enables the entire mobile gantry phenotype collection device to move on the track and switch between the horizontal (X) track and the vertical (Y) track.

[0083] It should be understood that in order to realize the automated movement of the gantry, cables can be installed on the main beam, end beam, and track portion of the gantry, and the power supply is used to control the movement of the mobile gantry phenotype collection device. Furthermore, the platform vehicle walking mechanism, the trolley walking mechanism, and the car walking mechanism are all electric devices, and the power supply is used to control the movement of the mobile gantry phenotype collection device in the Y-axis, X-axis, and Z-axis directions. Usually, the distance the gantry moves depends on the track. Those skilled in the art can adjust the speed and distance of the mobile gantry phenotype collection device in the Y-axis, X-axis, and Z-axis directions as needed.

[0084] In one or more specific embodiments, when the area of ​​the track portion is 20m wide (Y track) and 80m long (X track), the operating parameters of the mobile gantry phenotype collection device include: the operating speed of the X-axis trolley walking mechanism is: 0-10m / min, and the operating speed of the Y-axis platform car walking mechanism is: 0-10m / min. Further, the operating motor power of the X-axis trolley walking mechanism can be: 1.5KW×4 (servo motor); the operating motor power of the Y-axis platform car walking mechanism can be: 1KW×1 (servo motor).

[0085] It should be understood that GPS positioning (eg, GPS positioning software) can be used to adjust the specific positioning of the mobile gantry phenotype collection device, thereby accurately controlling the specific position of the device to collect phenotypes. In a specific embodiment, the positioning accuracy is: <1 mm.

[0086] In one or more embodiments, the detection device includes a trolley traveling mechanism, a manual pan-tilt platform connected to the trolley traveling mechanism, and an image acquisition device connected to the manual pan-tilt platform.

[0087] In one or more embodiments, the image acquisition device is detachably connected to the manual pan-tilt head, for example, by a slot or a screw fit. More specifically, in order to quickly and detachably connect the image acquisition device, the detection device may also be provided with an adapter plate, and the image acquisition device may be quickly fixed and disassembled by using the slot on the adapter plate.

[0088] In one or more embodiments, the manual pan / tilt head includes a sphere, a pan / tilt head main knob, a panoramic base knob, a 360° rotating base scale, a clamp base knob, a clamp base, and a sleeve.

[0089] In one or more embodiments, the clamp seat base knob and the clamp seat cooperate with each other, so that the tightness of the clamp seat can be adjusted, and is connected to the trolley travel mechanism.

[0090] In one or more embodiments, the ball connects the clamp mount and the sleeve together and cooperates with the panoramic base knob and the 360° rotation base scale to rotate the sleeve.

[0091] In one or more embodiments, the pan / tilt main knob is used to rotate the manual pan / tilt.

[0092] It should be understood that in order to realize the automated movement of the detection device, cables can be installed on the main beam of the gantry and the manual pan-tilt platform, and the detection device can be controlled to move on the main beam (Z-axis) track of the gantry using power supply.

[0093] In one or more embodiments, the image acquisition device includes a visible light camera, a multispectral camera, and an optional lighting device.

[0094] In one or more embodiments, in order to increase the clarity of image acquisition, the image acquisition device may include one or more (eg, 2, more than 3, more than 4, more than 5, or more) lighting devices.

[0095] In one or more embodiments, in order to prevent the image acquisition device from being overheated due to direct sunlight, the image acquisition device further includes a heat insulation board.

[0096] In one or more embodiments, in order to increase the accuracy of image acquisition, the image acquisition device may include multiple (eg, 2, more than 3, more than 4, more than 5, or more) cameras.

[0097] In one or more specific implementations, the image acquisition device is 5 RGB cameras and a multi-spectral camera. The image acquisition device is also called a sensor module.

[0098] In one or more embodiments, in order to increase the accuracy of image acquisition, the image acquisition device may include multiple (eg, 2, more than 3, more than 4, more than 5, or more) cameras.

[0099] In one or more implementation schemes, through camera module construction and communication protocol development, intervention in PLC host computer control protocol development, bridge communication construction, crop phenotyping equipment is integrated, a phenotypic data collection workstation is established, and the collection path of the crop trait collection equipment is planned to perform high-throughput automatic collection of field crop phenotypes.

[0100] In one or more embodiments, the image acquisition device is detachably connected to the clamping base of the manual pan / tilt head.

[0101] In one or more embodiments, the lens is detachably connected to the sleeve of the manual pan / tilt head.

[0102] Plant phenotypic data collection methods

[0103] The utility model provides a method for collecting plant phenotype data, comprising: using the mobile gantry phenotype collection device of the utility model to collect plant phenotype data.

[0104] In one or more embodiments, the method comprises:

[0105] 1) Controlling the mobile gantry phenotype collection device described in any embodiment of the utility model to move to a suitable position;

[0106] 2) Collect plant phenotypic images using a mobile gantry phenotypic collection device;

[0107] 3) Obtaining plant phenotypic data of the plant by analyzing the plant phenotypic images obtained in 2).

[0108] In one or more embodiments, the plant includes a crop plant, such as, but not limited to, rice, wheat, corn, soybean.

[0109] In one or more embodiments, the movement in step 1) includes movement in the directions of X-axis, Y-axis and Z-axis.

[0110] It should be understood that in step 1), in order to control the mobile gantry phenotype collection device to move to a suitable position, a control system (such as a PLC and a field industrial computer) can be installed in the mobile gantry phenotype collection device described in the utility model, and the software program of the control system can be used to control the movement of the mobile gantry phenotype collection device on the horizontal (X) track and the vertical (Y) track, and the movement of the detection device on the main beam (Z-axis) track of the gantry.

[0111] The utility model also provides an exemplary software program, including a server (PLC) main program and a field industrial computer main program.

[0112] The server (PLC) main program includes: a main thread and a collection thread; the main thread includes: sending a status inquiry command to the field industrial computer to inquire about the position of the mobile gantry phenotype collection device, thereby controlling whether the movement of the mobile gantry phenotype collection device stops; the collection thread includes: sending a collection command to the field industrial computer according to the collection path, receiving feedback from the field industrial computer, and if the picture already exists, the picture can be downloaded remotely or a picture clearing command can be sent to the field industrial computer, and the thread will be exited after the last collection point is collected.

[0113] The main program of the field industrial computer includes: receiving instructions from the server (PLC) main program, creating a new collection thread according to the collection instructions, sending an instruction to the server main program whether to stop collecting after collecting pictures, updating the real-time position of the mobile gantry phenotype collection device according to the status inquiry instruction, and clearing the picture according to the clear picture instruction; wherein the collection thread includes: updating the status and moving the mobile gantry phenotype collection device, updating the status and collecting images, updating the status and image processing, updating the status and saving pictures, and updating the status and exiting the thread.

[0114] In one or more embodiments, the functions of the server (PLC) main program include: controlling the reset, movement, and positioning of the mobile gantry phenotype collection device; controlling the light source switch of the lighting device; and controlling the moving speed of the mobile gantry phenotype collection device.

[0115] In one or more embodiments, the plant phenotype in step 2) includes: photos of the plant phenotype, the number of ears at the ear stage, and the projected area of ​​a single plant.

[0116] In one or more embodiments, in step 2), an image acquisition device of a mobile gantry phenotype acquisition device is used to collect pictures of plant phenotypes, and the overlap of pictures taken at two adjacent sampling points is more than 75%, more than 80%, more than 85%, more than 90% or higher.

[0117] It should be understood that the overlap of the photos taken at two adjacent sampling points determines the accuracy of the analysis in step 3). The higher the overlap, the higher the accuracy of the analysis in step 3). In addition, the accuracy of the analysis in step 3) is also related to the focal length and the number of sampling points of the image acquisition device. The smaller the focal length and the more sampling points, the higher the accuracy of the analysis in step 3). In one or more specific embodiments, when the plant is rice or wheat, since the leaf width of wheat and rice is much smaller than the leaf length, it is only necessary to ensure that the photo has enough sampling points in the leaf width direction, for example, the number of sampling points in the leaf width direction is >30, >40, >50 or more.

[0118] In one or more embodiments, the acquisition time (Ti) of a single sampling point can be calculated according to the distance (ΔL) moved by the mobile gantry phenotype acquisition device on the X-axis, the distance (ΔW) moved by the mobile gantry phenotype acquisition device on the Y-axis, the speed (v), the image acquisition time (Tshoot), the preprocessing time (Tmod), the saving time (Tsave), and the main program remote download time (Tload), as shown in Formula 1:

[0119] Ti=Tmove+Tshoot+Tmod+Tsave+Tload

[0120] Tmove=(ΔL+ΔW) / v

[0121] Tshoot+Tmod+Tsave≈3s

[0122] Tload≈5s(1)

[0123] In one or more embodiments, step 3) includes: using Pix4D software to process the plant phenotype images obtained in step 2), merging the images, and forming a three-dimensional reconstructed plant image.

[0124] In one or more embodiments, the analysis in step 3) includes: rice heading period analysis and rice single plant projected area analysis.

[0125] In one or more embodiments, the heading date analysis comprises:

[0126] 3.1) Manual labeling, including labeling the rice ear part of the rice phenotype pictures collected in step 2) (for example, using software such as Label studio and labelimg);

[0127] 3.2) Detection model construction: Yolo and rcnn models are used to train, test and verify the manually annotated images in 3.1) to build a detection model for heading period analysis;

[0128] 3.3) Application of the detection model: Analyze the heading period according to the detection model of 3.2); for example: according to the collection time point, obtain the dynamic curve of the number of rice ears with respect to time, define the date corresponding to the 10% maximum number of ears as the initial heading period, and combine the rice phenotype pictures collected in step 2) and the detection model of 3.2) to analyze the heading period.

[0129] In one or more embodiments, the rice plant projection area analysis includes:

[0130] 3.1) performing background denoising on the rice phenotype images collected in step 2);

[0131] 3.2) Distinguish rice from other plants, for example, use the CIVE vegetation index combined with the Otsu threshold to separate soil from vegetation, and remove duckweed and other targets that are smaller in area than the main body of rice by setting a threshold;

[0132] 3.2) Obtain the area covered by the rice plants and calculate the projected area of ​​a single rice plant = area covered by the rice plants ÷ number of rice plants.

[0133] The advantages of the utility model are:

[0134] 1) The mobile gantry phenotypic collection device of the utility model has both horizontal and vertical tracks, so it can be easily moved and expanded from one area to another for phenotypic collection.

[0135] 2) In the mobile gantry phenotype acquisition device of the utility model, the image acquisition device connected to the manual pan-tilt platform may include a camera and a lighting device, and may be adjusted according to the needs of image acquisition.

[0136] 3) In the mobile gantry phenotype collection device of the utility model, the detection device is detachable and easy to use.

[0137] The present invention is further described below in conjunction with specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. The specific conditions not specified in the following examples are usually based on conventional conditions or conditions recommended by the manufacturer.

[0138] Example 1: Mobile gantry phenotype collection device of the utility model

[0139] The utility model provides a mobile gantry phenotype collection device, comprising: a track portion (3) consisting of a track beam (1) and embedded parts (2) under the track beam, and a gantry (4) containing a detection device (5).

[0140] The track part (3) of the utility model is composed of a track beam (1) and an embedded part (2) under the track beam, and is the basic part for the movement of the mobile gantry, wherein the track beam (1) is composed of a track beam body (101) and a fixed part (102) and a slide rail (103) located on the track beam body (101). The embedded part (2) is composed of an embedded support part (201), a connecting part (202) and an embedded bracket (203), wherein the embedded support part (201) is 300 mm long, 200 mm wide and 10 mm thick. The embedded bracket (203) is 250 mm long. The embedded bracket (203) is embedded in the concrete foundation, usually in a hook shape, and is used to increase the stability of the embedded part. The concrete foundation is 500 mm wide and 500 mm high. The spacing between the embedded parts (2) is 600 mm. The diameter of the hook part of the embedded bracket (203) is 12 mm. The connecting component (202) connects the embedded support component (201) and the embedded bracket (203) together. The embedded support component (201) is used to support the track beam (1). The track portion (3) is divided into a horizontal (X) track and a vertical (Y) track. The moving wheels (405) of the mobile gantry can change direction to achieve switching between the horizontal (X) track and the vertical (Y) track. The area of ​​the track portion is usually consistent with the area to be detected. The utility model provides an exemplary track portion, namely an area with a width (Y track) of 20m and a length (X track) of 80m.

[0141] The gantry of the utility model comprises: an end beam (401), a main beam (Z) track (402), a lifting wheel group (403), a lifting wheel (404), a moving wheel (405), a platform vehicle running mechanism (406) and a trolley running mechanism (407); wherein the end beam (401) and the main beam (Z) track (402) are connected together, the end beam (401) plays a supporting role, usually in a triangular shape, and is used to support the entire gantry and maintain the stability of the gantry, and the span of the end beam is 3500mm. The main beam (Z) track (402) is parallel to the ground, and is detachably connected with a detection device (5). The platform vehicle running mechanism (406) is connected to one side of the end beam (401) to control the movement of the gantry (4) on the vertical (Y) track, and the trolley running mechanism (407) is connected to the other side of the end beam (401) to control the movement of the gantry (4) on the horizontal (X) track. The trolley travel mechanism (407) includes a drag chain electric tractor, which is used to tow the entire gantry (4) on the horizontal (X) track. The platform car travel mechanism (406) is a rack-type platform car, which drives the gantry (4) to move on the vertical (Y) track through a chain. The lifting wheel group (403) is connected to the lower end of the end beam (402) and is used to adjust the height of the main beam track (402) from the ground. The span of the lifting wheel group is 4000mm and the lifting height is 50mm. The lifting wheel group (403) is composed of a plurality of lifting wheels (404). The lower end of the lifting wheel group (403) is connected to a moving wheel (405), which allows the entire gantry to move on the track and can switch between the horizontal (X) track and the vertical (Y) track. It should be understood that in order to realize the automated movement of the gantry, cables can be installed on the main beam, end beam, and track portion of the gantry, and the power supply is used to control the movement of the mobile gantry phenotype collection device. Furthermore, the platform vehicle walking mechanism (406), the trolley walking mechanism (407), and the trolley walking mechanism (501) are all electric devices, and the power supply is used to control the movement of the mobile gantry phenotype collection device in the Y-axis, X-axis, and Z-axis directions. Usually, the distance the gantry moves depends on the track. Those skilled in the art can adjust the speed and distance of the mobile gantry phenotype collection device in the Y-axis, X-axis, and Z-axis directions as needed.The utility model provides an exemplary gantry, whose parameters include: a gantry span width of 20m, a lifting height of 5m, a maximum load of 0.2t, an X-axis stroke length of 85m (80m phenotype collection area + 5m maintenance area), an X-axis trolley main power supply drag chain electric traction device, a gantry power supply cable reel, an X-axis trolley running speed: 0-10m / min, an X-axis trolley running motor power: 1.5KW×4 (servo motor), a Y-axis stroke length of 20m, a Y-axis direction rack-type running platform vehicle, a Y-axis trolley running speed: 0-10m / min, a Y-axis trolley running motor power: 1KW×1 (servo motor), a Z-axis direction-a three-section electric telescopic hanger, a Z-axis direction intercepting trolley, a stroke of 3.5m, and automatic control walking in six directions of X, Y, and Z. The gantry's X, Y, and Z axes are integrated and debugged to meet the running speed of the X-axis trolley and the Y-axis trolley: 0-10m / min, and the positioning accuracy: <1mm.

[0142] The trolley travel mechanism (501) of the detection device (5) of the utility model is detachably and movably connected to the main beam (Z) track (402). The detection device (5) comprises a trolley travel mechanism (501), a manual pan-tilt head (502) connected to the trolley travel mechanism (501), and an image acquisition device (503) connected to the manual pan-tilt head (502). The manual pan-tilt head (502) comprises a sphere (601), a pan-tilt head main knob (602), a panoramic base knob (603), a 360° rotating base scale (604), a clamp base knob (605), a clamp seat (606) and a sleeve (607). The clamp base knob (605) and the clamp seat (606) cooperate with each other, so that the tightness of the clamp seat can be adjusted, and the clamp seat is connected to the trolley travel mechanism (501). The sphere (601) connects the clamping seat (606) and the sleeve (607) together, and cooperates with the panoramic base knob (603) and the 360° rotating base scale (604) to rotate the sleeve (607). The pan / tilt main knob (602) is used to rotate the manual pan / tilt (502). The image acquisition device (503) includes a visible light camera, a multi-spectral camera, and an illumination device. In order to increase the accuracy of image acquisition, the image acquisition device (503) may include multiple (e.g., 2, more than 3, more than 4, more than 5, or more) cameras. In order to increase the clarity of image acquisition, the image acquisition device (503) may include one or more (e.g., 2, more than 3, more than 4, more than 5, or more) illumination devices. By building a camera module and developing a communication protocol, intervening in the development of a PLC host computer control protocol, and building a bridge communication, crop phenotype collection equipment is integrated, a phenotype data collection workstation is established, and a collection path of the crop trait collection equipment is planned to perform high-throughput automatic collection of field crop phenotypes. The image collection device (503) is detachably connected to a clamping seat (606) of a manual pan-tilt platform (502), and the lens is detachably connected to a sleeve (607) of the manual pan-tilt platform (502).

[0143] Example 2: Plant phenotypic data collection method

[0144] This embodiment provides a method for collecting plant phenotypic data, comprising: using the mobile gantry phenotype collection device of the utility model to collect plant phenotypic data.

[0145] Specifically, the method includes:

[0146] 1) Controlling the mobile gantry phenotype collection device of the utility model to move to a suitable position;

[0147] 2) Collect plant phenotype images using a detection device;

[0148] 3) Obtaining plant phenotypic data of the plant by analyzing the plant phenotypic images obtained in 2).

[0149] More specifically, the mobile gantry phenotype acquisition device of step 1) includes an RGB camera and a multispectral camera. Step 2) includes: adding exif data to the image, writing information required for camera 3D reconstruction, transmitting the image back to the server via wireless network, receiving and saving the image on the server, and recording the image saving path.

[0150] Example 3, plant phenotypic data collection method parameters

[0151] 1. Development of crane control

[0152] The first step of fully automatic collection is to intervene in the crane control system. In this embodiment, the core of the mobile gantry phenotype collection device gantry (4) is a PLC, model: Siemens S7-1200, which has an Ethernet interface and can be easily connected to the local area network.

[0153] By programming the C# host computer through the existing API interface, the PLC memory can be read and written. Communicate with the Jingjiang Hoisting Equipment Factory to obtain the register address corresponding to the PLC function point. Modify the value in the function register to achieve the movement control of the gantry.

[0154] Functions to be implemented:

[0155] Reset, move and position of the vehicle (get / set)

[0156] Specify IO control (external light source switch get / set)

[0157] Driving speed setting (get / set)

[0158] 2. Sensor module development

[0159] ACamera Selection

[0160] The core of the image acquisition device (503) is 5 global exposure RGB cameras and multi-spectral cameras.

[0161] Global exposure ensures the synchronization of the camera acquisition process, avoiding defects such as image tearing and blurring caused by camera vibration and plant disturbance during camera shooting.

[0162] Considerations for lens selection: Since the leaf width of wheat and rice is much smaller than the leaf length, as long as the image has enough sampling points in the leaf width direction, sufficient details can be obtained for 3D reconstruction. Currently, the available lenses for this camera are 35mm and 50mm. The detailed information corresponding to the lenses is shown in Table 1.

[0163] Table 1 Accuracy, field of view, and number of sampling points of two lenses at different distances

[0164]

[0165] From Table 1, we can see that:

[0166] If the field of view covers at least one ridge (1.5*0.5m) within the working distance (2.5-5m), a lens with a focal length of 35mm should be used. At this time, the number of sampling points in the leaf width direction is >30, which is sufficient for 3D model reconstruction.

[0167] If higher-precision reconstruction is desired, a 50mm lens is used. In this case, the number of sampling points in the leaf width direction is >44, and the model accuracy is higher. However, the required acquisition interval needs to be reduced synchronously to ensure the overlap of adjacent images.

[0168] Considering that the higher the accuracy of plant 3D reconstruction, the better, it was decided to use a lens with a focal length of 50 mm, and by reducing the shooting distance, ensure that the overlap of adjacent photos is above 75%.

[0169] B Camera Attitude Adjustment Stand

[0170] The software Pix4D was tested for 3D reconstruction. The test results showed that the software can complete 3D reconstruction without knowing the exact position of the camera. It is speculated that the software can identify the same feature points from multiple pictures and automatically calculate the rigid body transformation matrix of the camera corresponding to the two pictures, thereby achieving 3D reconstruction.

[0171] Therefore, the camera does not need to be mounted on the electric pan-tilt head to accurately adjust the position and posture, but can be mounted on the pan-tilt head main knob (602), so that the operator can adjust the camera posture at any time, and lock the pan-tilt head main knob (602) after the adjustment is completed. Figure 4 shown.

[0172] C module structure design

[0173] The module is divided into two parts: the upper part is the power supply and switch part, the lower part is the sensor module, and the outer layer is wrapped in a sealed and waterproof box.

[0174] Input / output (IO) part: The upper input harness is a 220V power cord and a single network cable, which is connected to the module through a waterproof interface. The waterproof level must reach IP67 level.

[0175] The network cable and power cable are connected to the upper layer of the module, and are routed inside the module and directly connected to the camera;

[0176] The lower layer of the module is designed as a hollow cage structure, which reduces weight while leaving enough space for adjusting the camera posture;

[0177] The bottom of the module uses high-transmittance glass as the camera's visible light window. The glass can be opened from the bottom to facilitate adjustment of the camera's posture. Finally, the glass window is waterproof and sealed.

[0178] D module adapter board design

[0179] 1) In order to better control the camera, a module adapter plate is designed to be connected to the upper end of the camera of the image acquisition device (3);

[0180] 2) The adapter board is designed with a module bayonet, which allows the sensor module to be quickly plugged into the adapter board;

[0181] 3) The adapter board is designed with light source installation holes for fixing external light sources;

[0182] 4) A heat insulation board is designed on the adapter board and connected to the image acquisition device (3) to prevent direct sunlight from shining on the module, causing excessive temperature and damaging the module camera.

[0183] Intrinsic calibration of E camera module

[0184] Pix4D software can estimate the camera's intrinsic parameters and distortion coefficients from the input image, and save the intrinsic parameters and distortion coefficients to the camera model, thereby automatically calibrating and stitching pictures. The software supports the merging of multiple different camera pictures, which requires matching calculations on the pictures collected by different cameras.

[0185] In order to reduce the difficulty of image matching calculation, after acquiring the image data, the pre-calibrated camera internal parameters and distortion coefficients can be used to correct the distortion of the original image before handing it over to Pix4D for processing, thereby increasing the image matching accuracy and reducing the 3D reconstruction time.

[0186] In this embodiment, the checkerboard and Matlab camera calibration tools are used to calibrate the camera internal parameters and distortion coefficients, and the calibration results are stored in the configuration file for automatic call by the correction program.

[0187] 3. Development of field industrial control computers

[0188] AIndustrial computer installation

[0189] Furthermore, a PLC control cabinet can be arranged below the end beam (401), and the industrial computer can be installed in the PLC control cabinet, which can be quickly installed through the adapter plate. With the help of the PLC control cabinet shell and power supply line, additional wiring work is avoided, and it is more rainproof and lightning-proof.

[0190] B Industrial computer interface

[0191] Ethernet port: The industrial computer has dual Gigabit Ethernet ports. Network port 1 (LAN1) is wired to the camera module to stably collect camera images; network port 2 (LAN2) is wired to the bridge, and then bridged to the LAN where the server is located through the WIFI network of the wireless bridge.

[0192] Serial port: The industrial computer has its own RS232 serial port, which can be used for serial communication of the sensor.

[0193] USB interface: USB to Ethernet module can be used to connect to the Ethernet port of PLC.

[0194] C Industrial Computer Software Development

[0195] 1) Development of host computer for RGB camera

[0196] ① Based on the official SDK of Hikvision cameras, develop the camera's image acquisition program;

[0197] ② Based on the Opencv image recognition algorithm and calling the camera calibration parameters, the captured images are corrected for distortion;

[0198] ③ Write EXIF ​​information to the distortion-corrected image.

[0199] 2) Development of wireless image transmission program

[0200] ①Solution 1: Use FTP communication. The advantage of FTP communication is that it is highly efficient in transferring large files. By deploying an FTP server on the industrial computer and an FTP client on the server side of the computer room, file data can be transferred between the two PCs.

[0201] ②Solution 2: Windows shared folder. Using the shared folder, you can share the folder where the industrial computer saves camera pictures to the local area network. In the main program on the server PC, write a file stream reading program, enter the user name, password, and file path, and you can access the files in the shared folder.

[0202] 3)PLC control host computer development

[0203] Based on the third-party API (c#) of Siemens S7-1200PLC, and referring to the PLC point list provided by Jingjiang Hoisting Equipment Factory, the PLC host computer program was developed to realize the gantry's reset, movement, positioning, and light source switch functions.

[0204] 4) System framework development

[0205] The above three software modules need to provide command interfaces. When the main program framework receives valid commands, it will automatically call the corresponding software modules to implement the specified functions.

[0206] At the same time, the main program framework needs to respond to query commands from the remote end, feedback the execution status of each software module, and provide status flag signals for the main program running on the remote PC.

[0207] 4. Bridge Development

[0208] A Bridge Installation

[0209] Wireless bridges are installed in pairs: divided into "camera end" and "recorder end", and the two need to be installed opposite each other to ensure signal strength. The following work needs to be done:

[0210] An adapter needs to be designed to fix the "camera end" to the gantry beam and the "recorder end" outside the server room;

[0211] The LAN port of the "camera end" of the bridge is connected to the LAN port of the field industrial computer through a network cable; the LAN port of the "recorder end" of the bridge is connected to the LAN port of the server through a network cable;

[0212] B Bridge Bandwidth Test

[0213] Static testing

[0214] Take 5 to 10 preset points along the length of the gantry, copy the file over the wireless network, and measure the bandwidth of the gantry at each preset point.

[0215] Dynamic testing

[0216] During the movement of the gantry, the fluctuation of bandwidth is measured and the network stability is determined based on the number of packet losses.

[0217] 5Server Software Development

[0218] A Main program function framework construction

[0219] ① Design the "Functional Instruction Set" to send functional instructions to the main program of the field industrial computer, directing it to realize complex functions such as image acquisition, processing, driving control, and picture transmission.

[0220] ② Design the "Status Instruction Set" to send query instructions to the main program of the field industrial computer to obtain the operating status of all subsystems to optimize the system operation process.

[0221] B-image remote copy and clear

[0222] Remotely download the image cache data in the field industrial computer, delete the image cache after copying, and avoid the field industrial computer hard disk storage overflow.

[0223] Save the downloaded images to the specified folder according to the naming rules, and write the image name, save path, save time, scan batch, sensor ID and other information into the Excel file.

[0224] C Path Planning

[0225] The software can freely plan multiple experimental areas, input the scanning starting point, end point, interval, height and other information of each test area, the software can automatically calculate the optimal acquisition path, and send acquisition instructions to the field industrial computer according to the path, thereby realizing the image acquisition process.

[0226] D Collection process monitoring

[0227] 1) Real-time display of gantry position

[0228] 2) The status of each collection area is updated in real time: completed, collecting, waiting to be collected

[0229] 3) Collection progress display

[0230] E Image Classification Extraction

[0231] When using Pix4D to reconstruct a specified area with high precision, it is necessary to extract all images of the specified area from the hard disk. This function can be achieved by developing UI software. In the software, specify the scanning batch and GPS range of the photos to extract all photos that meet the search criteria and save them to the specified folder for 3D reconstruction.

[0232] Example 4: Plant phenotypic data collection process and efficiency estimation

[0233] 1. Collection process

[0234] This embodiment divides the collection process into two parts: the server main program and the field industrial computer main program. The former is responsible for planning and supervising the collection process, and the latter is responsible for executing the underlying sensor communication and control. Figure 5 shown.

[0235] 2. Single point acquisition time

[0236] The instruction communication time between the two programs can be ignored, and the main time is consumed in gantry movement, image acquisition, image processing, image storage, and remote download. The system acquisition time is shown in Formula 1.

[0237] In one or more embodiments, the acquisition time (Ti) of a single sampling point can be calculated based on (ΔL), (ΔW), and speed (v).

[0238] Ti=Tmove+Tshoot+Tmod+Tsave+Tload

[0239] Tmove=(ΔL+ΔW) / v

[0240] Tshoot+Tmod+Tsave≈3s

[0241] Tload≈5s (1)

[0242] Wherein, △L and △W are the moving distance of the mobile gantry phenotype acquisition device on the X axis (unit: m) and the moving distance of the mobile gantry phenotype acquisition device on the Y axis (unit: m), v represents the moving speed of the vehicle, the image acquisition time (Tshoot) includes the preprocessing time (Tmod) and the saving time (Tsave), the image acquisition time (Tshoot) is about 3s in total, and the main program remote download time (Tload) is about 5s.

[0243] It is estimated that when the vehicle moves at the maximum speed (10m / min), the minimum acquisition time for each point is:

[0244] T≈6*ΔD+8 (2)

[0245] Where △D is the moving interval, in meters.

[0246] 3. Estimation of total acquisition time

[0247] Estimate the time required for image acquisition of the entire experimental field (80*20m).

[0248] Considering that the RGB camera uses a 50mm focal length lens and an effective field of view of 26°×26°, in order to ensure the accuracy of image matching, the overlap of adjacent photos is calculated as 70%. The total number of sampling points at different working distances and the total acquisition time are shown in Table 2.

[0249] Table 2 Sampling time estimation table

[0250]

[0251] Note: Table 2 is an estimate of the collection time for the entire experimental field. The actual collection time is related to the planting density of the plants. When the plants are sparse, the collection time can be reduced by reasonably planning the route and avoiding collection in blank areas.

[0252] According to the number of collection points in Table 2, if each collection point saves 5 pictures, at least 109GB (3350 sampling points) of picture data is required, and at most 434GB (13400 sampling points) of storage space is required. It is recommended that the server storage space is greater than 5TB, which can support the storage requirements of 10 to 40 rounds of complete collection data. The actual storage space is related to the plant planting density. This example is an estimated value when the entire test field is fully sampled.

[0253] Example 5: Application of the mobile gantry phenotype collection device

[0254] In this embodiment, rice is taken as an example, and the mobile gantry phenotype acquisition device of the utility model is used for image acquisition and processing. The mobile gantry phenotype acquisition device of the utility model can cover and scan 1,600 square meters, and 481 varieties of single rice plants are planted in a rice garden (80 meters long and 20 meters wide), with 4 replicates for each variety, to collect growth phenotype data.

[0255] By taking photos with an RGB camera every day and performing photo processing and analysis, a series of phenotypic traits during the rice growth period can be obtained, including the number of rice panicles at the heading stage and the projected area of ​​a single rice plant.

[0256] Manual labeling uses open source software such as Label studio and labelimg to manually select rice ears. The manually labeled data set is divided into a training set, a validation set, and a test set with a ratio of 8:1:1. The target detection model, yolo series, rcnn series and other models are trained on the training set. The target detection model parameters are selected on the validation set. Finally, the accuracy of the model in identifying rice ears is evaluated on the test set.

[0257] After obtaining the rice panicle detection model, the dynamic curve of the number of rice panicles during the panicle period with respect to time can be obtained by using a mobile gantry phenotype collection device to shoot rice in units of days. Various heading periods of rice can be obtained by giving the corresponding date of the maximum panicle number percentage, and the date corresponding to 10% of the maximum panicle number is defined as the initial heading period. After comparing with the initial heading period traits observed manually, R 2 It reaches 0.9387, which shows that the heading period prediction results are highly consistent with the actual results.

[0258] In addition, the mobile gantry phenotype collection device of the utility model can also obtain the projection area of ​​a single rice plant. The exemplary method includes: firstly, background denoising is performed on the collected projection area phenotype of a single rice plant, the CIVE vegetation index is combined with the Otsu threshold to separate the soil from the vegetation, and the target such as duckweed, which is smaller than the main body of the rice, is removed by setting a threshold, thereby calculating the area covered by the rice plant.

[0259] The above-mentioned embodiments only express several implementation methods of the utility model, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the utility model. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the utility model, and these all belong to the protection scope of the utility model. Therefore, the protection scope of the patent of the utility model shall be subject to the attached claims. At the same time, all documents mentioned in the utility model are cited as references in this application, just as each document is cited as reference separately.

Claims

1. A mobile gantry phenotype collection device, characterized in that: The device comprises: a track portion consisting of a track beam, embedded parts under the track beam, and a gantry containing a detection device; The track section includes: a horizontal track and a vertical track; The gantry includes: an end beam, a main beam track, a lifting wheel group, a lifting wheel, a moving wheel, a platform car running mechanism and a trolley running mechanism; wherein the main beam track is parallel to the ground, a detection device is detachably and movably connected to the main beam track, and the detection device includes a trolley running mechanism for controlling the movement of the detection device on the main beam track; the end beam and the main beam track are connected together; the moving wheel can change direction to achieve switching between the horizontal track and the vertical track; the platform car running mechanism is connected to one side of the end beam to control the movement of the gantry on the vertical track; the trolley running mechanism is connected to the other side of the end beam to control the movement of the gantry on the horizontal track.

2. The mobile gantry phenotype collection device according to claim 1, characterized in that: The lifting wheel group is connected to the lower end of the end beam and is used to adjust the height of the main beam track from the ground; wherein the lifting wheel group is composed of a plurality of lifting wheels.

3. The mobile gantry phenotype collection device according to claim 1, characterized in that: The detection device also includes an adapter plate, and the detection device is fixed and disassembled by using a card slot on the adapter plate.

4. The mobile gantry phenotype collection device according to claim 1, characterized in that: The track beam is composed of a track beam body, a fixing component and a slide rail located on the track beam body.

5. The mobile gantry phenotype collection device according to claim 1, characterized in that: The embedded part is composed of an embedded supporting component, a connecting component and an embedded bracket, wherein the connecting component connects the embedded supporting component and the embedded bracket together.

6. The mobile gantry phenotype collection device according to claim 5, characterized in that: There are four pre-embedded brackets, which are respectively connected to four connecting parts of the pre-embedded supporting parts.

7. The mobile gantry phenotype collection device according to any one of claims 1 to 6, characterized in that: The detection device includes a trolley traveling mechanism, a manual pan-tilt platform connected to the trolley traveling mechanism, and an image acquisition device connected to the manual pan-tilt platform; wherein the image acquisition device and the manual pan-tilt platform are detachably connected together.

8. The mobile gantry phenotype collection device according to claim 7, characterized in that: The manual pan / tilt head comprises a sphere, a pan / tilt head main knob, a panoramic base knob, a 360° rotating base scale, a clamp base knob, a clamp base and a sleeve; the clamp base knob and the clamp base cooperate with each other to adjust the tightness of the clamp base, and are connected to the trolley travel mechanism; the sphere connects the clamp base and the sleeve together, and cooperates with the panoramic base knob and the 360° rotating base scale to rotate the sleeve; the pan / tilt head main knob is used to rotate the manual pan / tilt head.

9. The mobile gantry phenotype collection device according to claim 7, characterized in that: The image acquisition device includes a camera and an optional lighting device.

10. The mobile gantry phenotype collection device according to claim 9, characterized in that: The image acquisition device also includes a heat insulation plate.

Images