Vertical Agriculture Crop Phenotyping Data AGV Collection System

Abstract

The present invention relates to the field of monitoring technology, specifically to a vertical agricultural crop phenotypic data AGV acquisition system, including a track and an AGV trolley walking on the track, wherein the AGV trolley includes a mechanical system, a power system, a control system and an acquisition system; the power system is used to provide walking power for the mechanical system under the control of the control system. The AGV trolley of the present invention is equipped with retractable, multi-site acquisition equipment and high-definition resolution dual cameras (RGB+hyperspectral), which can adjust the horizontal and vertical directions of the camera to provide different shooting sites, and at the same time, a ranging sensor is installed on the camera side, which can adjust the height according to the change of plant height to ensure that the shooting height between the camera and the highest growth point of the crop is consistent, so as to realize multi-layer, multi-site phenotypic image acquisition of three-dimensional cultivation agriculture according to the specified route, timing and determined angle.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and specifically to a vertical agricultural crop phenotypic data AGV collection system, which has telescopic, multi-angle, and multi-modal collection functions and is suitable for vertical agricultural vegetable phenotypic data collection. Background Art

[0002] The traditional method of obtaining phenotypic parameters of the aboveground parts of plants is mainly based on manual measurement, using certain tools to manually measure specific parameters of plants in each area. In addition to being laborious, time-consuming, and unable to obtain a large amount of data in a short period of time, manual measurement also has defects such as large errors due to human subjective factors that seriously affect the accuracy and stability of the analysis results, may cause mechanical damage to plants, and interfere with normal plant growth. Therefore, manual measurement will gradually be replaced.

[0003] From the perspective of the mounting method, the existing plant phenotypic information collection devices include different forms such as platform type, conveyor belt type, gantry type, self-propelled type, vehicle-mounted type and hanging type; from the perspective of the applicable environment, they can be divided into two forms: field mobile type and fixed type in the facility. Among them, the plant phenotypic information collection devices for facilities are of two types: desktop type and conveyor belt type, according to the spatial characteristics of the specific working scene (climate chamber, growth room, plant factory or greenhouse). The plant phenotypic information collection platforms all use fixed imaging sensor positions, so that the target crops can be moved by means of conveyor belts and other mechanisms to collect image information. They need to be specifically designed and installed in combination with the greenhouse environment, and the construction cost is very high. Some economical and applicable greenhouses do not have the installation conditions at all, so they lack universal applicability.

[0004] Plant phenotypic information collection devices in facilities at home and abroad are mostly integrated into large platforms. They have high information collection environment control capabilities and high data accuracy, but they also have common problems such as large size, large floor space, and high investment and operating costs. Summary of the invention

[0005] The purpose of the present invention is to provide an AGV collection system for vertical agricultural crop phenotypic data to solve the problem mentioned in the above background technology that the existing technology cannot realize vertical agricultural phenotypic collection.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a vertical agricultural crop phenotypic data AGV collection system, comprising a track and an AGV trolley traveling on the track, characterized in that: the AGV trolley comprises a mechanical system, a power system, a control system and a collection system;

[0007] The power system is used to provide walking power for the mechanical system under the control of the control system; the acquisition system is used to control the mechanical system under the control of the control system to take pictures and collect pictures when the driving action of the power system reaches the photo point.

[0008] Preferably, the mechanical system includes a vehicle body and a safety device; the vehicle body is composed of a frame of steel structures, a drive device and a steering mechanism; the drive device drives the vehicle body to achieve all-round movement forward, backward or longitudinally, laterally, obliquely and rotationally under the action of the steering mechanism.

[0009] Preferably, the driving device is composed of a driving wheel, a reducer, a brake, a driving motor and a speed controller, the reducer, the brake and the speed controller are respectively connected to the driving motor, and the driving wheel is mounted on the output shaft of the reducer.

[0010] Preferably, the safety device is used to ensure the safety of the AGV during operation, as well as the safety of on-site personnel and various equipment; the safety device includes a non-contact anti-collision sensor and a contact anti-collision sensor installed at the front end of the vehicle body, and a conspicuous signal light and sound alarm device installed on the top of the vehicle body.

[0011] Preferably, the power system includes a battery pack and a charging device; the battery pack is installed on the vehicle body to power the entire AGV vehicle, the charging device includes a charging connector and an automatic charger, an automatic charger is installed at one end of the track, and a charging connector matching the automatic charger is installed at the bottom of the AGV vehicle, and the charging connector is electrically connected to the battery pack for charging it.

[0012] Preferably, the control system includes two parts: an on-board controller and a ground controller, and the on-board controller and the ground controller are electrically connected via a communication device;

[0013] The on-board controller includes an information transmission and processing device and a control device, and the information transmission and processing device and the control device are integrated and installed inside the vehicle body; the control device includes a drive control device, a steering control device and a safety control device, which are respectively used for controlling the drive, steering and safety.

[0014] Preferably, the communication device is used to realize information exchange between the AGV and the ground control station and ground monitoring equipment;

[0015] The information transmission and processing device is used to monitor the AGV, monitor the ground status of the AGV, and transmit information to the ground control station in real time.

[0016] Preferably, a guiding device is also provided on the track, and the guiding device is electrically connected to the control device;

[0017] The guiding device includes a magnetic permeability sensor and a track sensor, and the control device controls the AGV to move forward, backward, branch or exit according to the signals collected by the magnetic permeability sensor and the track sensor.

[0018] Preferably, the acquisition system includes a telescopic support device and a camera device;

[0019] Multiple telescopic bracket devices are fixed on the AGV cart. The telescopic bracket devices have a multi-layer telescopic structure and can achieve vertical height telescoping and horizontal telescoping. Each group of telescopic bracket devices includes two shooting positions, namely a vertical overhead shooting position and a horizontal side shooting position.

[0020] Preferably, the camera device is installed in an adjustable position at each shooting point of the telescopic bracket device. The camera device adopts a high-definition camera or a dual camera (RGB+hyperspectral). Both sides of the surface of the camera device are also equipped with distance sensors for detecting the distance value between the camera device and the shooting target. The camera device is controlled by the control device to take pictures according to the distance value.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention can solve the problem that the existing technology cannot realize vertical agricultural phenotyping collection. The AGV phenotyping collection device is equipped with a retractable bracket and a high-definition resolution camera at multiple locations, which can realize phenotyping image collection at different positions, heights, and angles, and automatically upload them to the server. The AGV also has a two-way data transmission function and can remotely adjust the collection direction.

[0023] It has the following advantages:

[0024] 1. Reduce labor costs. The AGV can automatically collect phenotypic data, replace manual work, and reduce labor costs.

[0025] 2. Improve the collection accuracy. The AGV car is equipped with a telescopic bracket and a multi-point collection camera, which can collect phenotypic data at different heights and angles, and automatically position according to preset parameters with high accuracy.

[0026] 3. Enhanced safety: The body of the AGV is equipped with various safety devices, such as emergency stop button, obstacle avoidance radar, anti-collision edge, etc., which provide safety for the working environment.

[0027] 4. With a high level of automation, the AGV can easily connect to other systems automatically, provide real-time execution inspection records, connect online with the greenhouse management system, and provide information to the factory management system.

[0028] 5. Suitable for use in complex occasions, the AGV car solves some special environmental problems that are not suitable for people to produce or work in, such as vertical cultivation, and the problem of inconvenience of assembly due to excessive height.

[0029] The present invention uses an AGV trolley equipped with scalable, multi-site acquisition equipment and a high-definition resolution camera, which can adjust the horizontal and vertical directions of the camera to provide different shooting locations. At the same time, a ranging sensor is installed on the camera side, which can adjust the height according to the change of plant height to ensure that the shooting height between the camera and the highest growth point of the crop is consistent, so as to realize multi-layer and multi-site phenotypic image acquisition of stereoscopic cultivation agriculture according to the specified route, timing, and determined angle. The phenotypic data in the vertical plant factory can be collected more comprehensively, and the accuracy and comprehensiveness of data collection can be improved. The collected images are uploaded to the analysis platform, analyzed by the phenotypic analysis module, and the phenotypic index parameters are output. The expert system is combined for diagnostic reasoning, and finally the decision-making suggestions are output.

[0030] Therefore, the present invention is easy to operate, low in cost, and can collect crop phenotype images from multiple angles and multiple positions for vertical agriculture crops. It can locate the image position and provide responsive environmental and water and fertilizer decision-making guidance for the collected images. It can solve the needs of vertical agriculture for both efficient use of planting space and good inspection and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a basic structural schematic diagram of the AGV phenotype collection system of the present invention;

[0032] Figure 2 It is a schematic diagram of the telescopic support device and the acquisition camera device of the present invention;

[0033] Figure 3 It is a schematic diagram of the AGV path planning system flow of the present invention;

[0034] Figure 4 This is a schematic diagram of the image recognition modeling process of the present invention;

[0035] Figure 5 It is a schematic diagram of the collection, analysis and decision-making workflow of the present invention. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] The structure of the AGV collection system for vertical agricultural crop phenotypic data provided by the present invention is as follows: Figure 1 As shown, it includes a track and an AGV car running on the track. The AGV car includes a mechanical system, a power system, a control system and a collection system.

[0038] The power system is used to provide the mechanical system with power for walking under the control of the control system; the acquisition system is used to control the mechanical system to take pictures and collect pictures when the power system drives the mechanical system to reach the photo point under the control of the control system.

[0039] Furthermore, if Figure 1 as well as Figure 3 As shown, the mechanical system includes a vehicle body and safety devices; the vehicle body is composed of a frame of steel structural parts, a driving device and a steering mechanism; the driving device drives the vehicle body to achieve all-round movement forward, backward or longitudinally, laterally, obliquely and rotationally under the action of the steering mechanism; the driving device is composed of a driving wheel, a reducer, a brake, a driving motor and a speed controller, the reducer, the brake and the speed controller are respectively connected to the driving motor, and the driving wheel is installed on the output shaft of the reducer; the safety device is used to ensure the safety of the AGV during operation, as well as the safety of on-site personnel and various equipment; the safety device includes a non-contact anti-collision sensor and a contact anti-collision sensor installed at the front end of the vehicle body, and a conspicuous signal light and sound alarm device installed on the top of the vehicle body.

[0040] Specifically, signal lights and sound alarm devices are used to alert surrounding operators. In addition, for AGV carts that need to run in both directions or move sideways, anti-collision sensors need to be installed on all four sides of the vehicle body. Once a fault occurs, the AGV cart will automatically give out sound and light alarms and use wireless communication to notify the AGV monitoring system.

[0041] Furthermore, if Figure 3 As shown, the power system includes a battery pack and a charging device; the battery pack is installed on the vehicle body to power the entire AGV vehicle, and the travel motor is fixedly connected to one side of the vehicle body to provide power for the travel of the AGV vehicle under the power supply of the battery pack; the charging device includes a charging connector and an automatic charger, an automatic charger is installed at one end of the track, and a charging connector matching the automatic charger is installed at the bottom of the AGV vehicle, and the charging connector is electrically connected to the battery pack for charging it.

[0042] Among them, the battery is a 24V or 48V industrial battery, and the fast charging is a high-current charging. The AGV itself has a charging limiter and a safety protection device. The charging device is arranged on the AGV trolley in the form of a ground electric shoe type or a wall-mounted type. Combined with the operating conditions of the AGV, the factors such as short circuits that may occur during operation are comprehensively considered, so as to consider configuring the AGV safety protection device. After the AGV runs to the charging position, the AGV charging connector is connected to the charging sliding touch plate of the automatic charger, and the maximum charging current can reach more than 200A.

[0043] Furthermore, if Figure 3 as well as Figure 4 As shown, the control system includes an on-board controller and a ground controller, and the on-board controller and the ground controller are electrically connected through a communication device; the on-board controller includes an information transmission and processing device and a control device, and the information transmission and processing device and the control device are integrated and installed inside the vehicle body; the control device includes a drive control device, a steering control device and a safety control device, which are respectively used for driving, steering and safety control work; the communication device is used to realize information exchange between the AGV trolley and the ground control station and the ground monitoring equipment; the information transmission and processing device is used to monitor the AGV trolley, monitor the ground status of the AGV, and transmit information with the ground control station in real time. A guiding device is also arranged on the track, and the guiding device is electrically connected to the control device; the guiding device includes a magnetic permeability sensor and a track sensor, and the control device controls the AGV trolley to move forward, backward, branch or exit according to the signals collected by the magnetic permeability sensor and the track sensor.

[0044] At present, both the on-board controller and the ground controller use microcomputers, which are connected by a communication device. The ground (outside the vehicle) controller issues control instructions, which are input into the on-board controller through the communication device to control the operation of the AGV; the on-board controller completes the manual control of the AGV, the activation of the safety device, the battery status, the steering limit, the brake release, the travel light, the drive and steering motor control, the monitoring of the charging contactor, and the driving safety monitoring. The ground controller completes the AGV scheduling, the issuance of control instructions, and the reception of AGV operation status information. The control system is the core of the AGV. The operation, monitoring, and various intelligent control of the AGV must be realized through the control system.

[0045] Furthermore, if Figure 1 as well as Figure 2As shown, the acquisition system includes a telescopic support device and a camera device; multiple telescopic support devices are fixed on the AGV trolley, and the telescopic support device has a multi-layer telescopic structure, which can realize vertical height telescopic and horizontal telescopic; each group of telescopic support devices includes two shooting positions, namely a vertical overhead shooting position and a horizontal side shooting position. The shooting horizontal and vertical distances can be set as required to ensure that when shooting in the vertical direction, the distance from the camera position to the highest growth point of the plant is consistent, and the distance from the horizontal shooting position to the plant stem is consistent, which is convenient for subsequent image recognition and eliminates the recognition offset caused by the distance; the camera device is installed at each shooting position of the telescopic support device with an adjustable position. The camera device adopts a high-definition camera or a dual camera (RGB+hyperspectral), and the two sides of the camera device surface are also equipped with distance sensors for detecting the distance value between the camera device and the shooting target, and the camera device is controlled by the control device to take pictures according to the distance value.

[0046] Path Planning System

[0047] like Figure 3 As shown in the figure, first determine the number of wireless APs that need to be deployed according to the size of the on-site space, and connect the on-board serial port server to the wireless AP to realize the communication between the AGV and the server; the server connects with other systems or call boxes, receives task calls, and assigns them to the dispatchable AGV; path planning sets the running path for each task. Traffic control records the location of the AGV and decides whether to release it based on the AGV's application for right of way and priority level. Charging management monitors the AGV power in real time. When the AGV power is lower than the set value, it returns to the standby area to receive the charging task after completing the task.

[0048] The AGV car downloads the route planning path from the server and saves it locally. If the server-side path is updated or added, it will be automatically downloaded and updated. Task scheduling accepts the task assignment from the server and executes it. The server receives the photo command, sets the path planning route according to the photo location, and the communication device sends the path planning route and the photo point to the information processing device of the AGV body. The information processing device sends the path planning route and the photo point to the control device. The driving device drives the motor under the control of the control device to drive the AGV body to move according to the path planning route. The steering device drives the AGV body to control the steering of the AGV body according to the path planning route under the control of the control device. When the AGV body reaches the photo point, the control device controls the telescopic bracket device to complete the determination of the photo location, and the camera device takes a photo of the photo location of the photo target. After the photo is taken, the photo result is uploaded to the server. The server will send the next photo target and the path planning route of the next photo target, and the AGV car will take a photo of the next target.

[0049] AGV carries a high-definition camera or dual cameras (RGB+hyperspectral) and a distance sensor, and an on-site photo control system. It controls the camera carried by the AGV through a Socket connection to take photos at a specified location and take screenshots through technologies such as FFmpeg\CV2. The image data is transferred to the local edge server via FTP and then uploaded to the cloud for storage. The location of the image (cultivation rack number, layer number, shooting location) and the shooting angle code (overhead shooting, side shooting, etc.) are submitted at the same time as the image business card information.

[0050] The cloud system records the corresponding relationship mapping of all AGV routes, photo points and cultivation racks. The system can locate the position of each picture on the cultivation rack, and then associate it with the crop information on the cultivation rack recorded in the production management system. After AI image recognition, the daily picture records combined with environmental data provide richer information for comprehensive monitoring of the crop growth process, which is one of the foundations supporting the feedback mechanism of intelligent planting strategies.

[0051] Remote control of AGV: There is a computer on site to control the AGV's actions (charging or operation), which can set the AGV's inspection speed, operation mode, inspection route, inspection rear-end position, charging / calibration position, and voice function prompt corpus selection.

[0052] Image positioning

[0053] The AGV phenotyping device is equipped with a multi-site high-definition resolution camera or dual cameras (RGB+hyperspectral), which takes photos of crops in vertical agriculture at a fixed time and route. The photo name is converted into a business card according to the X, Y, H, A, and B coordinates. The photo information is determined by the business card. X and Y determine the cultivation rack, H determines the height of the cultivation rack, and A and B represent the overhead and side view angles, respectively. The shooting point can be quickly identified through the business card coordinates, which is convenient for subsequent image recognition, analysis and comparison. At the same time, a distance sensor is installed next to the camera to detect the distance between the camera and the photo point; the AGV lithium battery powers the acquisition camera and the control board, and the photos are sent to the industrial computer for unified processing at one time through the AGV communication sensor; the scheduling system provides path planning and control for the AGV, and provides users with monitoring of the AP.

[0054] Image recognition modeling and phenotypic index parameter acquisition

[0055] like Figure 4As shown in the figure, AGV collects pictures taken at multiple locations and performs preprocessing, including denoising, cropping and standardization. It uses OpenCV image processing technology to extract and match feature points in multiple pictures, determine the correspondence between pictures, and use the estimated camera parameters to perform three-dimensional reconstruction of the matched feature points through triangulation. It uses methods such as the L-system algorithm to construct the branch structure and leaf model of the plant and generate a detailed three-dimensional plant model. Next, select a suitable machine learning or deep learning framework, such as TensorFlow or PyTorch, build an image recognition model, train the model using annotated training data sets, and adjust hyperparameters through the validation set to optimize model performance. The model can be used to extract phenotypic traits such as leaf morphological index extraction, RGB image extraction, and hyperspectral leaf stress conditions. When the image data is input, the model will automatically identify and output a three-dimensional model and corresponding phenotypic parameters. The phenotype of the plant can reflect the growth of the crop, such as plant water, nutrients, pests and diseases, biomass and quality, so that various phenotypic data with intrinsic connections can be accurately and quickly obtained, providing a basis for crop management decisions.

[0056] Expert system reasoning and decision making

[0057] like Figure 5 As shown in the figure, a set of rule bases and knowledge bases are formulated based on the knowledge of domain experts. The inference engine can imitate the decision-making process of experts through these rules, and integrate the image recognition model with the expert system, so that after the system recognizes the image, it can use the expert system to perform nutrient diagnosis, water diagnosis, and environmental condition diagnosis through the output phenotypic characteristic parameters. Provide feedback mechanism for the corresponding water and fertilizer irrigation system and environmental control system, and provide optimized irrigation strategy and environmental control strategy.

[0058] In summary, the present invention provides a vertical agricultural phenotypic acquisition device, which acquires images through a telescopic multi-site acquisition device and an AGV mobile device. The image acquisition module inputs the acquired images into the phenotypic analysis module, constructs a three-dimensional model, and extracts phenotypic data. The analyzed data is sent to the expert database decision module, and the phenotypic data is inferred in combination with the expert system. Finally, the system provides decision suggestions based on the reasoning results of the expert database.

[0059] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A vertical agricultural crop phenotypic data AGV collection system, including a track and an AGV vehicle running on the track, characterized in that: The AGV vehicle includes a mechanical system, a power system, a control system and a collection system; The power system is used to provide walking power for the mechanical system under the control of the control system; the acquisition system is used to control the mechanical system under the control of the control system to take pictures and collect pictures when the driving action of the power system reaches the photo point.

2. The vertical agricultural crop phenotypic data AGV collection system according to claim 1, characterized in that: The mechanical system includes a vehicle body and a safety device; the vehicle body is composed of a frame of steel structures, a driving device and a steering mechanism; the driving device drives the vehicle body to achieve all-round movement forward, backward or longitudinal, transverse, oblique and rotation under the action of the steering mechanism.

3. The vertical agricultural crop phenotypic data AGV collection system according to claim 2, characterized in that: The driving device is composed of a driving wheel, a reducer, a brake, a driving motor and a speed controller. The reducer, the brake and the speed controller are respectively connected to the driving motor, and the driving wheel is installed on the output shaft of the reducer.

4. The vertical agricultural crop phenotypic data AGV collection system according to claim 2, characterized in that: The safety device is used to ensure the safety of the AGV during operation, as well as the safety of on-site personnel and various equipment; the safety device includes a non-contact anti-collision sensor and a contact anti-collision sensor installed at the front end of the vehicle body, and a conspicuous signal light and sound alarm device installed on the top of the vehicle body.

5. The vertical agricultural crop phenotypic data AGV collection system according to claim 1, characterized in that: The power system includes a battery pack and a charging device; the battery pack is installed on the vehicle body to power the entire AGV vehicle, the charging device includes a charging connector and an automatic charger, an automatic charger is installed at one end of the track, and a charging connector matching the automatic charger is installed at the bottom of the AGV vehicle, and the charging connector is electrically connected to the battery pack for charging it.

6. The vertical agricultural crop phenotypic data AGV collection system according to claim 1, characterized in that: The control system includes two parts: an on-board controller and a ground controller, and the on-board controller and the ground controller are electrically connected via a communication device; The on-board controller includes an information transmission and processing device and a control device, and the information transmission and processing device and the control device are integrated and installed inside the vehicle body; the control device includes a drive control device, a steering control device and a safety control device, which are respectively used for controlling the drive, steering and safety.

7. The vertical agricultural crop phenotypic data AGV collection system according to claim 6, characterized in that: The communication device is used to realize information exchange between the AGV and the ground control station and ground monitoring equipment; The information transmission and processing device is used to monitor the AGV, monitor the ground status of the AGV, and transmit information to the ground control station in real time.

8. The vertical agricultural crop phenotypic data AGV collection system according to claim 1, characterized in that: A guiding device is also provided on the track, and the guiding device is electrically connected to the control device; The guiding device includes a magnetic permeability sensor and a track sensor, and the control device controls the AGV to move forward, backward, branch or exit according to the signals collected by the magnetic permeability sensor and the track sensor.

9. The vertical agricultural crop phenotypic data AGV collection system according to claim 1, characterized in that: The acquisition system includes a telescopic support device and a camera device; Multiple telescopic bracket devices are fixed on the AGV cart. The telescopic bracket devices have a multi-layer telescopic structure and can achieve vertical height telescoping and horizontal telescoping. Each group of telescopic bracket devices includes two shooting positions, namely a vertical overhead shooting position and a horizontal side shooting position.

10. The vertical agricultural crop phenotypic data AGV collection system according to claim 9, characterized in that: The camera device is installed at each shooting position of the telescopic bracket device. The camera device adopts a high-definition camera or a dual camera (RGB+hyperspectral). Both sides of the surface of the camera device are also equipped with distance sensors for detecting the distance value between the camera device and the shooting target. The camera device is controlled by the control device to take pictures according to the distance value.

Images