Device and method for acquiring multi-phenotypic character information of crops in facility environment

Abstract

The invention provides a facility environment crop multi-phenotypic character information acquisition device and method. The facility environment crop multi-phenotypic character information acquisition device comprises an imaging module and a terminal control module, the imaging module comprises a visible light camera, a thermal infrared camera, a multispectral camera and a depth camera; the visible light camera is used for acquiring crop visible light images and transmitting the images to the terminal control module; the thermal infrared camera is used for acquiring thermal infrared images of crops and transmitting the thermal infrared images to the terminal control module; the multispectral camera is used for acquiring crop spectral images and transmitting the crop spectral images to the terminal control module; the depth camera is used for acquiring a crop three-dimensional image and transmitting the image to the terminal control module; the terminal control module is connected with the imaging module and is used for processing an image acquired by the imaging module to obtain multi-modal image data containing crop multi-phenotypic character information, so that high-throughput, high-precision, automatic and damage-free acquisition of crop phenotypic numbers can be quickly realized, and the multi-modal image data is obtained; therefore, the phenotype of the plant can be comprehensively and accurately analyzed.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop multi-phenotype trait detection, and particularly relates to a device and method for obtaining multi-phenotype trait information of crops in a facility environment. Background Art

[0002] With the development of modern agricultural technology, crop phenomics has become a key field in agricultural scientific research and breeding. Crop phenotype refers to the external traits of organisms such as shape, structure, size, and color determined by genotype and environment, including morphological structure traits, physiological function traits, and component content traits of plants. These characteristics can reflect the genetic characteristics and environmental adaptability of crops and are widely used in crop breeding, crop biology research, etc. It is an important part of agricultural scientific research. Traditional methods for obtaining phenotypic data are often inefficient, costly, and difficult to achieve large-scale automation and high-throughput analysis. Summary of the Invention

[0003] In view of the above technical problems, the present invention provides a device and method for obtaining multi-phenotype trait information of crops in a facility environment to achieve the acquisition of multi-dimensional trait information of crops. Through the integrated design of multiple cameras and a terminal control module, it can meet the requirements of various platforms in the facility, has strong versatility, can perform non-destructive measurement on the habitat of crops inside the facility, and can simultaneously obtain morphological parameters, physiological and biochemical parameters of plants at the same time and the same position, obtaining multi-modal image data containing multi-phenotype trait information of crops, which is conducive to comprehensively and accurately analyzing the phenotype of plants and is conducive to solving the problems of continuous monitoring and phenotype analysis during the whole growth period of crops.

[0004] The present invention is a high-throughput information acquisition device that can quickly and accurately obtain crop phenotype data. According to different application carrier platforms, such as being mounted on, vehicle-mounted, field real-time monitoring, large indoor and outdoor automation platforms and other devices, it can quickly achieve high-throughput, high-precision, automated, and non-destructive acquisition of crop phenotype data. It is of great significance for improving crop breeding efficiency and agricultural production management level.

[0005] Note that the recording of these objectives does not prevent the existence of other objectives. One embodiment of the present invention does not need to achieve all the above objectives. Other objectives than the above can be extracted from the descriptions in the specification, drawings, and claims.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] A device for obtaining multi-phenotype trait information of crops in a facility environment includes an imaging module and a terminal control module;

[0008] The imaging module includes a visible light camera, a thermal infrared camera, a multispectral camera, and a depth camera; the visible light camera is used to acquire visible light images of crops and transmit them to the terminal control module; the thermal infrared camera is used to acquire thermal infrared images of crops and transmit them to the terminal control module; the multispectral camera is used to acquire spectral images of crops and transmit them to the terminal control module; the depth camera is used to acquire three-dimensional images of crops and transmit them to the terminal control module;

[0009] The terminal control module is connected to the imaging module, processes the images collected by the imaging module, and obtains multi-modal image data containing crop multi-phenotypic trait information.

[0010] In the above solution, a heat dissipation module is further included; the heat dissipation module includes a heat dissipation housing;

[0011] The visible light camera, the thermal infrared camera, the multispectral camera, and the depth camera are installed inside the heat dissipation housing. The lenses of the visible light camera, the thermal infrared camera, the multispectral camera, and the depth camera are all located on the bottom plate of the heat dissipation housing, and the lens positions of the visible light camera, the thermal infrared camera, the multispectral camera, and the depth camera respectively correspond to the lens holes on the bottom plate;

[0012] The multispectral camera is installed on one side of the center of the bottom plate, the depth camera is installed on the other side of the center of the bottom plate, opposite to the spectral camera, the visible light camera is installed on the left side of the depth camera, the thermal infrared camera is installed on the right side of the depth camera, and the visible light camera, the thermal infrared camera, the multispectral camera, and the depth camera are on the same horizontal plane and face the same direction;

[0013] The visible light camera is installed on a visible light camera bracket, and the visible light camera bracket is installed on the bottom plate;

[0014] The thermal infrared camera is installed on a thermal infrared camera bracket, and the thermal infrared camera bracket is installed on the bottom plate;

[0015] The multispectral camera is installed on a multispectral camera bracket, and the multispectral camera bracket is installed on the bottom plate;

[0016] The depth camera is installed on a depth camera bracket, and the depth camera fixed bracket is installed on the bottom plate;

[0017] Adjust the installation angles of the visible light camera bracket, the multispectral camera bracket, and the thermal infrared camera bracket respectively, so that the visible light camera focal plane of the visible light camera, the depth focal plane of the depth camera, and the thermal infrared focal plane of the thermal infrared camera are all located within the multispectral camera focal plane of the multispectral camera, and the crops are located at the center of the field of view during the acquisition process of the visible light camera, the thermal infrared camera, the multispectral camera, and the depth camera.

[0018] Furthermore, the terminal control module is installed inside the heat dissipation housing;

[0019] The terminal control module includes a control board, an extended network port, a wireless network card, and a voltage conversion module;

[0020] The control board and the voltage conversion module are installed on the first side plate of the heat dissipation housing and are isolated by insulating materials;

[0021] The voltage conversion module is installed on the first side plate of the heat dissipation housing through a voltage conversion module fixing bracket. The extended network port is connected to the control board. The wireless network card is installed on the outer wall of the heat dissipation housing and is communicatively connected to the control board. The control board and the imaging module are electrically connected to the voltage conversion module respectively to convert alternating current into direct current. The control board is communicatively connected to the imaging module.

[0022] In the above solution, a cooling fan and a temperature sensor are further provided in the heat dissipation module;

[0023] The cooling fan and the temperature sensor are respectively connected to the terminal control module; heat dissipation guide holes are opened at both ends of the heat dissipation housing;

[0024] The temperature sensor collects the temperature information inside the heat dissipation housing and transmits it to the terminal control module; the terminal control module compares the collected temperature information with the preset working temperature value of the imaging module and controls the turning on of the cooling fan and the imaging module according to the comparison result;

[0025] The preset working value of the imaging module is T1 to T2;

[0026] When the temperature information collected by the temperature sensor is lower than the lower limit value T1, the terminal control module controls the cooling fan to stop working. When the collected temperature is T1 to T2, the terminal control module starts the cooling fan; when the cooling fan is running, if the temperature information collected by the temperature sensor still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working, realizing high-temperature warning inside the device and protecting the imaging sensor.

[0027] In the above solution, handles are installed at both ends of the heat dissipation housing, and the installed handles are located above the outward extension position of the heat dissipation guide holes;

[0028] There are buckle installation holes at the top of the heat dissipation housing, and a detachable buckle is installed on the buckle installation holes.

[0029] In the above solution, an interface connection module is further included; the interface connection module includes interface connection terminals and connection cables;

[0030] The interface connection terminals are installed on the heat dissipation housing and are located below the heat dissipation guide holes. The interface connection terminals are connected to the terminal control module through connection cables.

[0031] A control method for an apparatus for obtaining multi-phenotypic trait information of crops in a facility environment, comprising the following steps:

[0032] The visible light camera acquires visible light images of the crops and transmits them to the terminal control module; the thermal infrared camera acquires thermal infrared images of the crops and transmits them to the terminal control module; the multi-spectral camera acquires spectral images of the crops and transmits them to the terminal control module; the depth camera acquires three-dimensional images of the crops and transmits them to the terminal control module;

[0033] The terminal control module performs image analysis on the obtained visible light images to obtain the color, texture, and high-resolution morphological features of the crops, and obtains visible light information on the morphological parameters of the crops, including leaf area, plant height, growth trajectory, and leaf morphology;

[0034] The terminal control module performs image analysis on the obtained thermal infrared images to obtain thermal infrared information including an infrared temperature distribution feature of the crops, which is used to detect the physiological and water stress information of the crops;

[0035] The terminal control module performs image analysis on the obtained multi-spectral images to obtain spectral information including leaf color texture and hierarchical structure features, and the spectral information is used to evaluate the physiological and biochemical state of the crops;

[0036] The terminal control module performs three-dimensional point cloud reconstruction processing on the obtained depth images to obtain the external contour, plant morphology, and geometric parameters of the crops, which are used to analyze the crop canopy, plant height, and volume features;

[0037] The terminal control module performs atlas fusion on the morphological parameters of the crops, the morphological parameters of the crops, the leaf color texture, the hierarchical structure features, and the reconstructed three-dimensional morphological information of the crops. After the atlas fusion, a single image contains multi-modal image data of the visible light information, thermal infrared information, spectral information, and three-dimensional point cloud of the crops.

[0038] In the above solution, the following steps are further included:

[0039] Time synchronization: The terminal control module receives a synchronization trigger instruction sent by the host computer, generates a unified high-precision pulse signal, and distributes it to the visible light camera, thermal infrared camera, multi-spectral camera, and depth camera via the terminal control module; after receiving the signal, the visible light camera, thermal infrared camera, multi-spectral camera, and depth camera immediately start data acquisition and attach a timestamp to each frame of data, so that the multi-phenotypic trait information is aligned in the time dimension;

[0040] Spatial synchronization: Before data acquisition, it is necessary to perform spatial calibration on the cameras using a checkerboard. The visible light camera, thermal infrared camera, multispectral camera, and depth camera are used to capture the checkerboard below respectively, and the relative pose relationship of the checkerboard corner points is identified and calculated. Based on the relative pose relationship of the checkerboard corner points and combined with the captured data of the corresponding checkerboard by the visible light camera, thermal infrared camera, multispectral camera, and depth camera, the external parameter pose relationship between the visible light camera, thermal infrared camera, multispectral camera, and depth camera is generated. Finally, a unified spatial coordinate system for the visible light camera, thermal infrared camera, multispectral camera, and depth camera is established to achieve spatial alignment of multimodal data.

[0041] In the above solution, the following steps are also included:

[0042] On the basis of completing time synchronization and spatial synchronization, the obtained multimodal image data specifically includes spectral map fusion and two-dimensional to three-dimensional multi-scale registration;

[0043] The specific process of the spectral map fusion is as follows:

[0044] The RGB images captured by the visible light camera, the multispectral data captured by the multispectral camera, and the thermal imaging matrix captured by the thermal infrared camera are preprocessed, including denoising, radiometric correction, and pixel-level alignment. Then, based on the pose relationship between the visible light camera, thermal infrared camera, multispectral camera, and depth camera generated by spatio-temporal synchronization, the data is projected onto a unified spatial coordinate system;

[0045] A weighted fusion algorithm is used to integrate the texture information of the RGB images, the band characteristics of the multispectral data, and the temperature distribution of the thermal imaging matrix to form a multimodal spectral map:

[0046] The RGB images, multispectral data, and thermal imaging data are normalized to obtain a texture feature matrix, a band feature matrix, and a temperature distribution matrix;

[0047] Based on correlation calculation, weights are assigned to the color texture feature matrix, spectral feature matrix, and temperature distribution matrix to form a set of weighting coefficients;

[0048] The weighted fusion algorithm is used to fuse the preprocessed multimodal data matrix to generate a multimodal spectral map containing color texture, spectrum, and temperature information;

[0049] Among them, the fusion formula of the weighted fusion algorithm is: F = W1T + W2S + W3R

[0050] In the formula, T is the temperature distribution matrix, S is the band feature matrix, R is the color texture feature matrix, W1 is the temperature weighting coefficient, W2 is the spectral weighting coefficient, W3 is the color texture weighting coefficient, and W1 + W2 + W3 = 1;

[0051] Image - Point Cloud Fusion: Reconstruct the three - dimensional point cloud of crops using the point cloud data collected by a depth camera; extract the edge feature points of leaves in RGB images, multispectral images, and thermal infrared images, and through the Iterative Closest Point (ICP) algorithm using the Iterative Nearest Point method, perform initial registration of the two - dimensional image data with the three - dimensional point cloud; adopt a multi - scale optimization strategy to accurately register the pixel - level local details and align the crop structure at the overall scale; then normalize the collected thermal imaging matrix to obtain the temperature distribution matrix, assign weights to it based on correlation calculations, and finally integrate the temperature distribution matrix with the texture feature matrix and the band feature matrix using a weighted fusion algorithm to form a three - dimensional multimodal map containing color texture, spectral, and temperature information, realizing the fusion of images and point clouds.

[0052] In the above - mentioned solution, the following steps are also included:

[0053] The temperature sensor collects the temperature information inside the heat - dissipation housing and transmits it to the terminal control module;

[0054] When the temperature information collected by the temperature sensor is lower than the lower limit value T1, the terminal control module controls the cooling fan to stop working. When the collected temperature is between T1 and T2, the terminal control module starts the cooling fan; when the cooling fan is running, if the temperature information collected by the temperature sensor still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1. The present invention adopts an integrated multi - source imaging camera to synchronously obtain multi - dimensional phenotypic information such as the morphological structure of crops, spectral reflection characteristics, and canopy temperature distribution in a single operation, breaking through the limitation of traditional single cameras that can only obtain limited traits, achieving full - dimensional coverage from the analysis of plant three - dimensional structures to the quantification of biochemical components and the perception of water stress states, and providing a comprehensive data basis for the comprehensive analysis of various crop traits.

[0057] 2. The present invention generates a unified high - precision pulse signal through the control board to drive the multi - camera synchronous acquisition, eliminates the time jitter problem caused by software triggering in a dynamic scenario, ensures that visible light, thermal infrared, multispectral, and depth data are strictly aligned to the same timestamp, avoids data temporal misalignment caused by camera response delays, and overcomes the problem of inconsistent image information caused by plant shaking, providing a reliable basis for multi - source data fusion.

[0058] 3. Through the optimized design of multi-camera collaborative layout and spatial calibration, the present invention integrates visible light, thermal infrared, multi-spectral and depth cameras on the same substrate in a symmetric distribution manner at the hardware level, and can adjust the angles of the camera brackets so that their focal planes are coplanar and the centers of the fields of view are aligned, ensuring full coverage acquisition of the same crop area by the multi-modal cameras and eliminating the problem of field of view deviation caused by installation position deviation of traditional split devices; further combining spatial calibration to automatically calculate the external parameter matrices of each camera and establish a unified spatial coordinate system, realizing two-dimensional to three-dimensional spatial matching of multi-source heterogeneous cameras, and providing spatially consistent data support for multi-dimensional analysis and precise fusion of crop phenotypic characteristics.

[0059] 4. Through multi-modal atlas fusion and image-point cloud registration technology, the present invention fuses visible light images, multi-spectral data, thermal infrared temperatures and depth data of depth cameras at the pixel level to generate spatio-temporally unified multi-modal data, and further combines point cloud reconstruction technology to realize precise mapping of image and point cloud information, achieving "image-atlas-point cloud" integrated data fusion of image-atlas integration and image-point cloud integration, and providing multi-source multi-modal high-dimensional data for dynamic monitoring of crop growth.

[0060] 5. The dynamic heat dissipation regulation mechanism of the present invention maintains the constant temperature operation of the imaging module in the high-temperature and high-humidity environment of the facility greenhouse through real-time monitoring and intelligent temperature control, reduces the influence of temperature difference fluctuations inside and outside the greenhouse on the camera sensitivity, and ensures the stable acquisition of image data such as visible light and multi-spectral; at the same time, when the heat dissipation fan is running, if the temperature information collected by the temperature sensor still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working. By setting such temperature warnings, the service life of the device in high-temperature scenarios is extended, providing continuous and reliable data support for the analysis of the growth state of greenhouse crops, temperature warnings and environmental regulation.

[0061] 6. Through the dual-mode drive design of integral and decentralized modes, the present invention integrates the multi-modal camera array into a flexibly switchable independent device: in the integral drive mode, the control board uniformly schedules all cameras to achieve one-key start / stop and parameter synchronization configuration, meeting the high-efficiency requirements of large-scale continuous monitoring in the greenhouse; in the decentralized drive mode, visible light cameras, thermal infrared cameras, multi-spectral cameras and depth cameras can be independently enabled, which is beneficial to adapt to specific phenotypic data acquisition on platforms such as gantry cranes and assembly lines. Through the design of standardized interfaces and general communication protocols, the device can be quickly deployed on different platforms without modifying the existing structure, significantly reducing the hardware transformation cost in multiple scenarios, and at the same time retaining the flexibility of multi-camera collaboration or independent operation, providing highly compatible and plug-and-play hardware support for large-scale monitoring and precise diagnosis in facility agriculture.

[0062] Note that the description of these effects does not preclude the existence of other effects. One embodiment of the present invention does not necessarily have to have all of the above effects. Effects other than the above can be obviously seen and extracted from the descriptions in the specification, drawings, claims, etc. Brief Description of the Drawings

[0063] Figure 1 is a schematic diagram of the overall external shape of the device for obtaining multi-phenotypic trait information of crops in a facility environment according to an embodiment of the present invention;

[0064] Figure 2 is a schematic diagram of the imaging focal plane adjustment of the device for obtaining multi-phenotypic trait information of crops in a facility environment according to an embodiment of the present invention;

[0065] Figure 3 is an internal structure diagram of the device for obtaining multi-phenotypic trait information of crops in a facility environment according to an embodiment of the present invention;

[0066] Figure 4 is an internal structure diagram of the device for obtaining multi-phenotypic trait information of crops in a facility environment according to an embodiment of the present invention from another angle;

[0067] Figure 5 is a schematic diagram of the bottom structure of the device for obtaining multi-phenotypic trait information of crops in a facility environment according to an embodiment of the present invention;

[0068] In the figure, 1. heat dissipation housing, 2. visible light camera bracket, 3. visible light camera, 4. depth camera, 5. multi-spectral camera, 6. thermal infrared camera, 7. thermal infrared camera bracket, 8. transformer module fixing bracket, 9. transformer module, 10. control board, 11. interface connection terminal, 12. heat dissipation guide hole, 13. handle, 14. buckle installation hole, 15. detachable buckle, 16. depth camera fixing bracket, 17. cross beam bracket, 18. extended network port, 19. first side plate, 20. second side plate, 21. bottom plate, 22. multi-spectral camera bracket, 23. wireless network card, 24. connection cable, 25. screw, 26. fan, 27. temperature sensor, 28. dust filter, 29. multi-spectral focal plane, 30. visible light focal plane, 31. depth focal plane, 32. thermal infrared focal plane. Detailed Embodiment

[0069] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0070] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "front", "rear", "left", "right", "upper", "lower", "axial", "radial", "vertical", "horizontal", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, the meaning of "a plurality" is two or more, unless otherwise specifically defined.

[0071] In the present invention, unless otherwise clearly specified and defined, the terms "mounted", "connected", "coupled", "fixed", etc. shall be construed in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] As Figures 1-5 shown is a preferred embodiment of the device for acquiring multi-phenotypic trait information of crops in the facility environment according to the present invention. The device for acquiring multi-phenotypic trait information of crops in the facility environment includes an imaging module and a terminal control module; the imaging module includes a visible light camera 3, a thermal infrared camera 6, a multispectral camera 5, and a depth camera 4; the visible light camera 3 is used to acquire visible light images of crops and transmit them to the terminal control module; the thermal infrared camera 6 is used to acquire thermal infrared images of crops and transmit them to the terminal control module; the multispectral camera 5 is used to acquire spectral images of crops and transmit them to the terminal control module; the depth camera 4 is used to acquire three-dimensional images of crops and transmit them to the terminal control module;

[0073] The terminal control module is connected to the imaging module and processes the images collected by the imaging module to obtain multi-modal image data containing multi-phenotypic trait information of crops.

[0074] In a specific embodiment of the present invention, a heat dissipation module is further included; the heat dissipation module includes a heat dissipation housing 1; preferably, the heat dissipation housing 1 is a metal heat dissipation housing; the heat dissipation housing 1 includes a first side plate 19, a second side plate 20 disposed opposite to the first side plate, a front plate, a rear plate, a bottom plate 21, and a top plate;

[0075] The visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4 are installed inside the heat dissipation housing 1. The lenses of the visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4 are all located on the bottom plate 21 of the heat dissipation housing 1, and the lens positions of the visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4 respectively correspond to the lens holes on the bottom plate 21;

[0076] The multispectral camera 5 is installed on one side of the center of the bottom plate 21, the depth camera 4 is installed on the other side of the center of the bottom plate 21, opposite to the spectral camera 5, the visible light camera 3 is installed on the left side of the depth camera 4, and the thermal infrared camera 6 is installed on the right side of the depth camera 4. The visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4 are on the same horizontal plane and face the same direction;

[0077] The visible light camera 3 is installed on the visible light camera bracket 2, and the visible light camera bracket 2 is installed on the bottom plate 21 through screws 25;

[0078] The thermal infrared camera 6 is installed on the thermal infrared camera bracket 7, and the thermal infrared camera bracket 7 is installed on the bottom plate 21 through screws 25;

[0079] The multispectral camera 5 is installed on the multispectral camera bracket 22, and the multispectral camera bracket 22 is installed on the bottom plate 21 through screws 25;

[0080] The depth camera 4 is installed on the depth camera bracket 16, and the depth camera fixing bracket 16 is installed on the bottom plate 21 through screws 25;

[0081] The visible light camera 3, the thermal infrared camera 6, and the depth camera 4 are installed at a certain inclination angle, specifically as follows:

[0082] Taking the downward target plane of the multispectral camera 5 as a reference, according to crops at different heights, the installation angles of the visible light camera bracket 2, the multispectral camera bracket 22, and the thermal infrared camera bracket 7 are respectively adjusted so that the visible light camera focal plane 30 of the visible light camera 3, the depth focal plane 31 of the depth camera 4, and the thermal infrared focal plane 32 of the thermal infrared camera 6 are all located within the multispectral camera focal plane 29 of the multispectral camera 5, so that the crops are located in the center of the field of view during the acquisition process of the visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4, as Figure 2 shown, to ensure the best imaging effect.

[0083] As Figure 3As shown in the figure, the terminal control module is installed in the heat dissipation housing 1; the terminal control module includes a control board 10, an expansion network port 18, a wireless network card 23, and a voltage conversion module 9; the control board 10 and the voltage conversion module 9 are installed on the first side plate 19 of the heat dissipation housing 1 and are isolated by insulating materials; the voltage conversion module 9 is installed on the first side plate 19 of the heat dissipation housing 1 through a voltage conversion module fixing bracket 8, the expansion network port 18 is connected to the control board 10, cross beam brackets 17 are respectively provided at the joints of the first side plate 19 and the second side plate 20, the wireless network card 23 is installed on the outer wall of the cross beam bracket 17 of the heat dissipation housing 1, the wireless network card 23 is communicatively connected to the control board 10, the control board 10 and the imaging module are respectively electrically connected to the voltage conversion module 9 to convert alternating current into direct current, and the control board 10 is communicatively connected to the imaging module. Through the control board 10, the present invention generates a unified high-precision pulse signal to drive multi-camera synchronous acquisition, eliminates the time jitter problem caused by software triggering in a dynamic scenario, ensures that visible light, thermal infrared, multi-spectral, and depth data are strictly aligned to the same timestamp, avoids data time-domain misalignment caused by camera response delay, overcomes the problem of inconsistent image information caused by plant shaking, and provides a reliable basis for multi-source data fusion.

[0084] A heat dissipation fan 26 and a temperature sensor 27 are further provided in the heat dissipation module for adjusting the temperature inside the heat dissipation housing 1; the heat dissipation fan 26 and the temperature sensor 27 are respectively connected to the terminal control module; heat dissipation guide holes 12 are opened at both ends of the heat dissipation housing 1; preferably, the heat dissipation guide holes 12 adopt a microchannel structure with a pore diameter of 0.5 mm to optimize air circulation and heat dissipation efficiency, and a dust-proof net 28 is installed in the heat dissipation guide holes 12 to prevent external dust from entering the device. The temperature sensor 27 collects the temperature information inside the heat dissipation housing 1 and transmits it to the terminal control module; the terminal control module compares the collected temperature information with the preset working temperature value of the imaging module and controls the start of the heat dissipation fan 26 and the imaging module according to the comparison result;

[0085] The preset working value of the imaging module is T1 to T2; where T1 and T2 are set according to actual requirements.

[0086] When the temperature information collected by the temperature sensor 27 is lower than the lower limit value T1, the terminal control module controls the heat dissipation fan 26 to stop working. When the collected temperature is T1 to T2, the terminal control module starts the heat dissipation fan 26 to adjust the internal temperature of the system; when the heat dissipation fan 26 is running, if the temperature information collected by the temperature sensor 27 still continuously exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working, realizing high-temperature warning inside the device and protecting the imaging camera.

[0087] Preferably, the heat dissipation fan 26 is installed on the control board 10 and the heat dissipation guide holes 12, and the connection surface is filled with a heat-conducting medium, and the heat-conducting medium is heat-conducting silicone grease.

[0088] As Figure 4 shown, preferably, handles 13 are installed at both ends of the heat dissipation housing 1, and the handles 13 are installed above the positions where the heat dissipation guide holes 12 extend outward.

[0089] As Figure 5 shown, there are detachable buckle mounting holes 14 at the top of the heat dissipation housing 1, and the detachable buckle 15 is installed on the buckle mounting holes 14.

[0090] The device for obtaining multi-phenotypic trait information of facility environment crops further includes an interface connection module; the interface connection module includes interface connection terminals 11 and connection cables 24; the interface connection terminals 11 are installed on the heat dissipation housing 1 and are located below the heat dissipation guide holes 12, and the interface connection terminals 11 are connected to the terminal control module through the connection cables 24; the interface connection terminals 11 are used to connect to external devices such as monitors, mice, and communication controls for convenient interactive processing.

[0091] The device for obtaining multi-phenotypic trait information of facility environment crops can collect crop images while processing the images collected by the imaging module to obtain multi-modal image data containing crop multi-phenotypic trait information. The control board 10 drives the imaging module to work to realize information acquisition and analysis to obtain multi-modal image data.

[0092] Preferably, the present invention can be conveniently installed on a fixed or mobile phenotypic platform through the detachable buckle 15, and the control board 10 supports offline or real-time analysis of RGB images, multi-spectral data, thermal imaging matrices, and depth point clouds to realize the entire information acquisition and analysis, specifically including the following:

[0093] The imaging module collects multi-source data of the crop canopy or side view, and sends command information through the terminal control module to control the information collection of the imaging module, and transmits the information back to the control board 10 for phenotypic analysis of the crops; the control board 10 of the terminal control module performs visual display on the upper computer through an external monitor, keyboard, and mouse, controls the imaging module to perform parallel collection of crop multi-phenotypic trait information by sending commands, transmits the obtained multi-source data back to the control board 10, and can be immediately displayed on the upper computer to complete the collection of multi-phenotypic trait information.

[0094] The parallel collection of crop multi-phenotypic traits by the imaging module is specifically as follows:

[0095] Using the port output of the control board 10, a control signal is generated according to the command sent by the host computer. The control signal is transmitted to the imaging module via the terminal control module as a hardware trigger signal; the hardware trigger signal drives the visible light camera 3, the thermal infrared camera 6, the multispectral camera 5, and the depth camera 4 in the imaging module to start synchronous acquisition, so as to ensure the precise alignment of multi-phenotypic trait data in time; the visible light image of the crop is obtained by the visible light camera 3, the thermal infrared image of the crop is obtained by the thermal infrared camera 6, the multispectral image of the crop is obtained by the multispectral camera 5, and the depth image of the crop is obtained by the depth camera 4.

[0096] After receiving the hardware trigger signal, the imaging module acquires multi-modal image data in parallel and transmits the acquired data to the terminal control module for processing and storage; among them, the control board 10 ensures that each camera completes synchronous acquisition within the same trigger cycle through a preset timing logic or pulse signal, so as to realize the efficient parallel acquisition of crop multi-phenotypic trait information.

[0097] The imaging module of the present invention includes a visible light camera 3, a thermal infrared camera 6, a multispectral camera 5, and a depth camera 4, which comprehensively collect crop phenotypic information. The terminal control module is equipped with a control board 10 for controlling the imaging module and real-time collecting multi-phenotypic information of the crop. The interface connection module realizes stable connection with external devices and is used to display and control the terminal control module. The heat dissipation module contains the imaging module and the terminal control module inside. Through the integration of a three-fan heat dissipation structure inside the heat dissipation shell 1, the system stability is increased. The imaging module of the present invention provides an efficient and reliable tool for crop phenotypic research by integrating a variety of imaging technologies, and promotes crop improvement and optimized cultivation management.

[0098] A control method for a device for obtaining multi-phenotypic trait information of crops in a facility environment includes the following steps:

[0099] The visible light camera 3 acquires the visible light image of the crop and transmits it to the terminal control module; the thermal infrared camera 6 acquires the thermal infrared image of the crop and transmits it to the terminal control module; the multispectral camera 5 acquires the spectral image of the crop and transmits it to the terminal control module; the depth camera 4 acquires the three-dimensional image of the crop and transmits it to the terminal control module;

[0100] The terminal control module performs image analysis on the obtained visible light image to obtain the color, texture, and high-resolution morphological features of the crop, and obtains visible light information such as morphological parameters of the crop including leaf area, plant height, growth trajectory, and leaf morphology;

[0101] The terminal control module performs image analysis on the obtained thermal infrared image to obtain thermal infrared information including an infrared temperature distribution feature of the crop, which is used to detect the physiological and water stress information of the crop;

[0102] The terminal control module performs image analysis on the acquired multispectral images to obtain spectral information including leaf color texture and hierarchical structure features, and the spectral information is used to evaluate the physiological and biochemical status of crops;

[0103] The terminal control module performs three-dimensional point cloud reconstruction processing on the acquired depth images to obtain the external contours, plant morphology, and geometric parameters of the crops, which are used to analyze the crop canopy width, plant height, and volume characteristics;

[0104] The terminal control module performs atlas fusion on the morphological parameters of the crops, the morphological parameters of the crops, leaf color texture, hierarchical structure features, and the reconstructed three-dimensional topography information of the crops. After the atlas fusion, a single image contains multi-modal image data of the visible light information, thermal infrared information, spectral information, and three-dimensional point cloud of the crops.

[0105] The control method of the facility environment crop multi-phenotype trait information acquisition device further includes the following steps:

[0106] Time synchronization: The control board 10 of the terminal control module receives the synchronous trigger instruction sent by the host computer, generates a unified high-precision pulse signal, and distributes it to the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 via the terminal control module; after receiving this signal, the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 immediately start data acquisition and attach a timestamp to each frame of data to ensure the precise alignment of multi-phenotype trait information in the time dimension;

[0107] Spatial synchronization: Before data acquisition, it is necessary to perform spatial calibration on the cameras through a checkerboard. The visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 photograph the checkerboard below, identify and calculate the relative pose relationship of the checkerboard corner points; based on the relative pose relationship of the checkerboard corner points, combined with the shooting data of the corresponding checkerboard by the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4, an external parameter pose relationship between the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 is generated using a calibration algorithm, and finally a unified spatial coordinate system of the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 is established to achieve the spatial alignment of multi-modal data.

[0108] Whiteboard calibration: Under the condition of consistent environmental information, a multispectral standard whiteboard with a known reflectivity is selected and placed at the center of the field of view of the multispectral camera 5. Adjust the exposure time, gain and other parameters of the spectral camera 5, and perform multiple groups of shootings on each band under the same environmental conditions, and record relevant information. Transmit the shooting data to the control board 10, extract the pixel values of the whiteboard area, and calculate the average value. According to the relationship between the known reflectivity and the whiteboard pixel values, calculate the reflectivity of the later crops.

[0109] The control method of the device for acquiring multi-phenotypic trait information of facility environment crops further includes the following steps:

[0110] On the basis of completing time synchronization and spatial synchronization, the acquired multi-modal image data specifically includes atlas fusion and two-dimensional to three-dimensional multi-scale registration;

[0111] The specific atlas fusion is as follows:

[0112] Preprocess the RGB images collected by the visible light camera 3, the multi-spectral data collected by the multi-spectral camera 5, and the thermal imaging matrix collected by the thermal infrared camera 6, including denoising, radiometric correction, and pixel-level alignment; then, based on the pose relationships between the visible light camera 3, the thermal infrared camera 6, the multi-spectral camera 5, and the depth camera 4 generated by spatio-temporal synchronization, project the data onto a unified spatial coordinate system;

[0113] Adopt a weighted fusion algorithm to integrate the texture information of the RGB images, the band features of the multi-spectral data, and the temperature distribution of the thermal imaging matrix to form a multi-modal atlas:

[0114] Normalize the RGB images, multi-spectral data, and thermal imaging data to obtain a texture feature matrix, a band feature matrix, and a temperature distribution matrix;

[0115] Based on correlation calculation, assign weights to the color texture feature matrix, the spectral feature matrix, and the temperature distribution matrix to form a set of weighting coefficients;

[0116] Use the weighted fusion algorithm to fuse the preprocessed multi-modal data matrix to generate a multi-modal atlas containing color texture, spectral, and temperature information;

[0117] Among them, the fusion formula of the weighted fusion algorithm is: F = W1T + W2S + W3R

[0118] In the formula, T is the temperature distribution matrix, S is the band feature matrix, R is the color texture feature matrix, W1 is the temperature weighting coefficient, W2 is the spectral weighting coefficient, W3 is the color texture weighting coefficient and satisfies W1 + W2 + W3 = 1;

[0119] Through the optimized design of multi-camera collaborative layout and spatial calibration, at the hardware level, visible light, thermal infrared, multispectral, and depth cameras are integrated on the same substrate in a symmetric distribution manner. The angles of the camera brackets can be adjusted to make their focal planes coplanar and the centers of the fields of view aligned, ensuring full coverage acquisition of the same crop area by the multi-modal cameras and eliminating the problem of field-of-view deviation caused by installation position deviation of traditional split devices. Further, by automatically calculating the external parameter matrices of each camera through spatial calibration and establishing a unified spatial coordinate system, two-dimensional to three-dimensional spatial matching of multi-source heterogeneous cameras is achieved, providing spatially consistent data support for multi-dimensional analysis and precise fusion of crop phenotypic characteristics.

[0120] Image-point cloud fusion: Use the point cloud data collected by the depth camera 4 to reconstruct the three-dimensional point cloud of the crop; extract key feature points such as the leaf edges in the RGB image, multispectral image, and thermal infrared image, and through the iterative closest point, use the ICP algorithm to perform initial registration of the two-dimensional image data and the three-dimensional point cloud; adopt a multi-scale optimization strategy to accurately register the pixel-level local details and align the crop structure at the overall scale; then normalize the collected thermal imaging matrix to obtain the temperature distribution matrix, assign weights to it based on correlation calculation, and finally use the weighted fusion algorithm to integrate the temperature distribution matrix with the texture feature matrix and the band feature matrix to form a three-dimensional multi-modal map containing color texture, spectral, and temperature information, realizing the fusion of the image and the point cloud.

[0121] Through multi-modal map fusion and image-point cloud registration technology, this invention pixel-level fuses visible light images, multispectral data, thermal infrared temperatures, and depth camera depth data to generate spatio-temporally unified multi-modal data. Further combined with point cloud reconstruction technology, it realizes the precise mapping of image and point cloud information, achieving the "image-map-point cloud" integrated data fusion of image-map integration and image-point cloud integration, providing multi-source and multi-modal high-dimensional data for crop growth dynamic monitoring.

[0122] Through the dual-mode drive design of integral and decentralized types, the multi-modal camera array is integrated into a flexibly switchable independent device: In the integral drive mode, the control board 10 uniformly schedules all cameras to achieve one-key start / stop and parameter synchronization configuration, meeting the high-efficiency requirements of large-scale continuous monitoring in greenhouses; in the decentralized drive mode, the visible light camera 3, thermal infrared camera 6, multispectral camera 5, and depth camera 4 can be independently enabled, which is conducive to adapting to platforms such as gantry cranes and assembly lines for directional acquisition of specific phenotypic data. Through the design of standardized interfaces and general communication protocols, this device can be quickly deployed on different platforms without modifying the existing structure, significantly reducing the hardware transformation cost in multiple scenarios. At the same time, it retains the flexibility of multi-camera collaboration or independent operation, providing highly compatible and plug-and-play hardware support for large-scale monitoring and precise diagnosis in facility agriculture.

[0123] The control method of the device for acquiring multi-phenotypic trait information of crops in the facility environment further includes the following steps:

[0124] The temperature sensor 27 collects the temperature information inside the heat dissipation housing 1 and transmits it to the terminal control module;

[0125] When the temperature information collected by the temperature sensor 27 is lower than the lower limit value T1, the terminal control module controls the heat dissipation fan 26 to stop working. When the collected temperature is between T1 and T2, the terminal control module starts the heat dissipation fan 26 to adjust the internal temperature of the system. When the heat dissipation fan 26 is running, if the temperature information collected by the temperature sensor 27 still continuously exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working to protect the system safety.

[0126] The dynamic heat dissipation regulation mechanism of the present invention maintains the constant temperature operation of the imaging module in the high-temperature and high-humidity environment of the facility greenhouse through real-time monitoring and intelligent temperature control, reduces the influence of the temperature difference fluctuation between inside and outside the greenhouse on the camera sensitivity, and ensures the stable acquisition of image data such as visible light and multi-spectral. At the same time, when the heat dissipation fan is running, if the temperature information collected by the temperature sensor still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working. By setting such temperature warnings, the service life of the equipment in high-temperature scenarios is extended, providing continuous and reliable data support for the analysis of the growth state of greenhouse crops, temperature warnings, and environmental regulation.

[0127] The device for acquiring multi-phenotypic trait information of crops in the facility environment can operate as an integrated system, that is, the device can operate as an independent sensor, and can also perform independent data analysis, that is, independently drive the independent acquisition and analysis of information of a single camera. The specific implementation is as follows:

[0128] Integrated system working mode: Integrate the visible light camera 3, depth camera 4, characteristic spectral camera 5, and thermal infrared camera 6, and realize the collaborative control and real-time data spatio-temporal synchronization of multiple cameras through the built-in control board 10. The data flows of each camera pass through the unified CAN bus communication interface, and through the integrated drive mode, the acquisition, preprocessing, and storage are completed under the spatio-temporal synchronization framework, forming an integrated multi-modal data output.

[0129] Independent data analysis mode: Equipped with an independent data processing unit, supporting offline or real-time analysis of RGB images, multi-spectral data, thermal imaging matrices, and depth point clouds. Each camera can operate independently, analyze the corresponding data respectively, complete the extraction of the plant shape, and output parameters such as the leaf area index and plant height of the crop phenotypic traits.

[0130] The information acquisition of the device for acquiring multi-phenotypic trait information of crops in the facility environment has the functions of overexposure and underexposure reminders. The specific implementation is as follows:

[0131] During the device information collection process, the image data collected by visible light cameras, feature spectral cameras, etc. is obtained in real time. Using image analysis algorithms, the average value of image pixels is calculated. The system preset overexposure and underexposure pixel average value thresholds, as well as high and low brightness pixel ratio thresholds. If the pixel average value is higher than the overexposure threshold and the high brightness pixel ratio exceeds the corresponding threshold, overexposure is determined; if it is lower than the underexposure threshold and the low brightness pixel ratio exceeds the corresponding threshold, underexposure is determined. When overexposure or underexposure is determined, the collection interface will prompt with eye-catching colors and text so that the operator can adjust in time.

[0132] Further, in a specific embodiment of the present invention, in view of the problems of complex light environment in the facility environment and the existence of light differences on the crop surface, the prior art single-channel image local shadow adaptive compensation algorithm based on non-linear mapping of lightness features is adopted. The coefficient of variation of the lightness distribution of the image before and after image compensation is reduced from 40.96% to 13.50%, a reduction of 27.46%; based on the lightness compensation algorithm, a lightness-saturation-hue compensation relationship is constructed to realize local shadow compensation of color images.

[0133] In terms of single-channel image shadow compensation, first, the collected color image is converted into a grayscale image, and a grayscale distribution curve is generated according to the pixel lightness values in the grayscale image; then, based on the statistical characteristics of pixel lightness, the dark area in the image is identified, and a non-linear mapping model of dark part - bright part, that is, the lightness compensation curve, is generated in real time based on the percentage curve cut-off; finally, the generated lightness compensation curve is used to remap the pixels in the grayscale image to improve the brightness of the dark area and make the brightness of the whole image more uniform.

[0134] In terms of color image shadow compensation, first, the collected color image is decomposed into three channels: hue, lightness, and saturation, and based on the lightness compensation curve obtained in the single-channel compensation algorithm, a regional mapping relationship is established with the lightness feature as the benchmark; then, the dark area is identified in the lightness channel, and non-linear mapping functions of hue and saturation features, that is, the hue compensation curve and the saturation compensation curve, are respectively established based on the regional mapping relationship; finally, the generated hue compensation curve and saturation compensation curve are used to remap the pixels in the color image, adjust the hue and saturation of the dark area, realize the compensation of the color image shadow, and improve the color uniformity and visual effect of the image.

[0135] The acquisition method of the present invention effectively improves the image acquisition quality through ambient light adaptive compensation. The single-channel compensation algorithm improves the brightness of the dark part of the grayscale image and makes the overall brightness of the image uniform; the color compensation algorithm adjusts the hue and saturation of the dark part, enhances the color uniformity and visual effect, avoids overexposure or underexposure, and significantly improves the data availability in low-light or high-light scenarios.

[0136] The present invention adopts an integrated multi-source imaging camera to synchronously obtain multi-dimensional phenotypic information such as the morphological structure of crops, spectral reflection characteristics, and canopy temperature distribution in a single operation, breaking through the limitation that traditional single cameras can only obtain limited traits, achieving full-dimensional coverage from the analysis of plant three-dimensional structures to the quantification of biochemical components and the perception of water stress states, and providing a comprehensive data basis for the comprehensive analysis of various crop traits.

[0137] Furthermore, the present invention can effectively reduce signal crosstalk during the collaborative operation of multiple sensors by optimizing the internal electrical layout, using hierarchical routing, signal isolation, and anti-interference techniques for low-voltage power lines (power supply lines for devices such as cameras and control boards) and signal lines (communication lines for cameras and control boards), ensuring the stable operation of visible light, thermal infrared, multispectral, and depth cameras in complex environments with high humidity and multiple devices such as greenhouse facilities, avoiding distortion or frame loss problems of image data caused by electromagnetic interference, and significantly improving the consistency and reliability of multi-modal data acquisition; at the same time, the compact cable layout reduces the space occupied inside the device, reduces the risk of failures caused by wire abrasion and short circuits, enhances the durability of the device during continuous operation, and provides long-term stable hardware support for precise greenhouse management.

[0138] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0139] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present invention, and they are not intended to limit the protection scope of the present invention. Any equivalent embodiments or changes made without departing from the technical spirit of the present invention should be included within the protection scope of the present invention.

Claims

1. An apparatus for obtaining multi-phenotypic trait information of crops in a facility environment, characterized in that, It includes an imaging module and a terminal control module; The imaging module includes a visible light camera (3), a thermal infrared camera (6), a multispectral camera (5), and a depth camera (4); the visible light camera (3) is used to acquire visible light images of crops and transmit them to the terminal control module; the thermal infrared camera (6) is used to acquire thermal infrared images of crops and transmit them to the terminal control module; the multispectral camera (5) is used to acquire spectral images of crops and transmit them to the terminal control module; the depth camera (4) is used to acquire three-dimensional images of crops and transmit them to the terminal control module; The terminal control module is connected to the imaging module and processes the images collected by the imaging module to obtain multimodal image data containing multi-phenotypic trait information of crops.

2. The facility environment crop multi-phenotypic trait information acquisition device according to claim 1, wherein It also includes a heat dissipation module; the heat dissipation module includes a heat dissipation housing (1); The visible light camera (3), the thermal infrared camera (6), the multispectral camera (5), and the depth camera (4) are installed inside the heat dissipation housing (1). The lenses of the visible light camera (3), the thermal infrared camera (6), the multispectral camera (5), and the depth camera (4) are all located on the bottom plate (21) of the heat dissipation housing (1), and the lens positions of the visible light camera (3), the thermal infrared camera (6), the multispectral camera (5), and the depth camera (4) respectively correspond to the lens holes on the bottom plate (21); The multispectral camera (5) is installed on one side of the center of the bottom plate (21), the depth camera (4) is installed on the other side of the center of the bottom plate (21), opposite to the spectral camera (5). The visible light camera (3) is installed on the left side of the depth camera (4), and the thermal infrared camera (6) is installed on the right side of the depth camera (4). The visible light camera (3), the thermal infrared camera (6), the multispectral camera (5), and the depth camera (4) are on the same horizontal plane and face the same direction; The visible light camera (3) is installed on the visible light camera bracket (2), and the visible light camera bracket (2) is installed on the bottom plate (21); The thermal infrared camera (6) is installed on the thermal infrared camera bracket (7), and the thermal infrared camera bracket (7) is installed on the bottom plate (21); The multispectral camera (5) is installed on the multispectral camera bracket (22), and the multispectral camera bracket (22) is installed on the bottom plate (21); The depth camera (4) is installed on the depth camera bracket (16), and the depth camera fixed bracket (16) is installed on the bottom plate (21); Adjust the installation angles of the visible light camera bracket (2), the multispectral camera bracket (22), and the thermal infrared camera bracket (7) respectively, so that the visible light camera focal plane (30) of the visible light camera (3), the depth focal plane (31) of the depth camera (4), and the thermal infrared focal plane (32) of the thermal infrared camera (6) are all located within the multispectral camera focal plane (29) of the multispectral camera (5), so that the visible light camera (3), the thermal infrared camera (6), the multispectral camera (5), and the depth camera (4) can achieve that the crops are located in the center of the field of view during the acquisition process.

3. The device for obtaining multi-phenotypic trait information of facility environment crops according to claim 2, characterized in that, The terminal control module is installed inside the heat dissipation housing (1); The terminal control module includes a control board (10), an extended network interface (18), a wireless network card (23), and a voltage conversion module (9); The control board (10) and the voltage conversion module (9) are installed on the first side plate (19) of the heat dissipation housing (1) and are isolated by insulating materials; The voltage conversion module (9) is installed on the first side plate (19) of the heat dissipation housing (1) through a voltage conversion module fixing bracket (8). The extended network interface (18) is connected to the control board (10). The wireless network card (23) is installed on the outer wall of the heat dissipation housing (1), and the wireless network card (23) is communicatively connected to the control board (10). The control board (10) and the imaging module are electrically connected to the voltage conversion module (9) respectively to convert alternating current into direct current. The control board (10) is communicatively connected to the imaging module.

4. The device for obtaining multi-phenotypic trait information of facility environment crops according to claim 2, wherein A heat dissipation fan (26) and a temperature sensor (27) are further provided in the heat dissipation module; The heat dissipation fan (26) and the temperature sensor (27) are respectively connected to the terminal control module; heat dissipation guide holes (12) are opened at both ends of the heat dissipation housing (1); The temperature sensor (27) collects the temperature information inside the heat dissipation housing (1) and transmits it to the terminal control module; the terminal control module compares the collected temperature information with the preset working temperature value of the imaging module and controls the turning on of the heat dissipation fan (26) and the imaging module according to the comparison result; The preset working value of the imaging module is T1 to T2; When the temperature information collected by the temperature sensor (27) is lower than the lower limit value T1, the terminal control module controls the heat dissipation fan (26) to stop working. When the collected temperature is T1 to T2, the terminal control module starts the heat dissipation fan (26); when the heat dissipation fan (26) is running, if the temperature information collected by the temperature sensor (27) still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working.

5. The device for obtaining multi-phenotypic trait information of facility environment crops according to claim 2, characterized in that, Handles (13) are installed at both ends of the heat dissipation housing (1), and the handles (13) are installed above the positions where the heat dissipation guide holes (12) extend outward; There are buckle mounting holes (14) at the top of the heat dissipation housing (1), and a detachable buckle (15) is installed in the buckle mounting holes (14).

6. The device for obtaining multi-phenotypic trait information of facility environment crops according to claim 4, wherein It further includes an interface connection module; the interface connection module includes interface connection terminals (11) and connection cables (24); The interface connection terminals (11) are installed on the heat dissipation housing (1) and are located below the heat dissipation guide holes (12). The interface connection terminals (11) are connected to the terminal control module through the connection cables (24).

7. A control method for an apparatus for acquiring multi-phenotypic trait information of facilities environment crops according to any one of claims 1-6, characterized in that, It includes the following steps: The visible light camera (3) acquires a visible light image of the crop and transmits it to the terminal control module; the thermal infrared camera (6) acquires a thermal infrared image of the crop and transmits it to the terminal control module; the multispectral camera (5) acquires a spectral image of the crop and transmits it to the terminal control module; the depth camera (4) acquires a three-dimensional image of the crop and transmits it to the terminal control module; The terminal control module performs image analysis on the acquired visible light image to obtain the color, texture, and high-resolution morphological features of the crop, and obtains the morphological parameters of the crop, including visible light information such as leaf area, plant height, growth trajectory, and leaf morphology; The terminal control module performs image analysis on the acquired thermal infrared image to obtain thermal infrared information including an infrared temperature distribution feature of the crop, which is used to detect the physiological and water stress information of the crop; The terminal control module performs image analysis on the acquired multispectral image to obtain spectral information including leaf color texture and hierarchical structure features, and the spectral information is used to evaluate the physiological and biochemical state of the crop; The terminal control module performs three-dimensional point cloud reconstruction processing on the acquired depth image to obtain the external contour, plant morphology, and geometric parameters of the crop, which are used to analyze the crop canopy, plant height, and volume features; The terminal control module performs atlas fusion on the morphological parameters of the crop, the morphological parameters of the crop, leaf color texture, hierarchical structure features, and the reconstructed three-dimensional morphological information of the crop. After atlas fusion, a single image contains multimodal image data of the visible light information, thermal infrared information, spectral information, and three-dimensional point cloud of the crop.

8. The control method of the device for obtaining multi-phenotypic trait information of facility environment crops according to claim 7, characterized in that, It also includes the following steps: Time synchronization: The terminal control module receives the synchronous trigger instruction sent by the host computer, generates a unified high-precision pulse signal, and distributes it to the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4) via the terminal control module; after receiving this signal, the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4) immediately start data acquisition and attach a timestamp to each frame of data, so that the multi-phenotypic trait information is aligned in the time dimension; Spatial synchronization: Before data acquisition, it is necessary to perform spatial calibration on the cameras through a checkerboard. The visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4) photograph the checkerboard below, identify and calculate the relative pose relationship of the checkerboard corner points; based on the relative pose relationship of the checkerboard corner points, combined with the shooting data of the corresponding checkerboard by the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4), generate the external parameter pose relationship between the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4), and finally establish a unified spatial coordinate system for the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4) to achieve spatial alignment of multimodal data.

9. The control method of the device for obtaining multi-phenotypic trait information of facility environment crops according to claim 8, characterized in that, It also includes the following steps: On the basis of completing time synchronization and spatial synchronization, the acquired multimodal image data specifically includes atlas fusion and image-point cloud fusion; The specific process of the atlas fusion is as follows: Preprocess the RGB image acquired by the visible light camera (3), the multispectral data acquired by the multispectral camera (5), and the thermal imaging matrix acquired by the thermal infrared camera (6), including denoising, radiometric correction, and pixel-level alignment; Then, based on the pose relationships among the visible light camera (3), thermal infrared camera (6), multispectral camera (5), and depth camera (4) generated by spatio-temporal synchronization, the data is projected onto a unified spatial coordinate system; The weighted fusion algorithm is used to integrate the texture information of the RGB image, the band characteristics of the multispectral data, and the temperature distribution of the thermal imaging matrix to form a multi-modal map: The RGB image, multispectral data, and thermal imaging data are normalized to obtain a texture feature matrix, a band feature matrix, and a temperature distribution matrix; Based on correlation calculation, weights are assigned to the color texture feature matrix, spectral feature matrix, and temperature distribution matrix to form a set of weighted coefficients; The weighted fusion algorithm is used to fuse the preprocessed multi-modal data matrices to generate a multi-modal map containing color texture, spectral, and temperature information; Among them, the fusion formula of the weighted fusion algorithm is: F = W1T + W2S + W3R In the formula, T is the temperature distribution matrix, S is the band feature matrix, R is the color texture feature matrix, W1 is the temperature weighted coefficient, W2 is the spectral weighted coefficient, and W3 is the color texture weighted coefficient, and W1 + W2 + W3 = 1; Image-point cloud fusion: The 3D point cloud of the crop is reconstructed using the point cloud data collected by the depth camera (4); the leaf edge feature points in the RGB image, thermal infrared image, and multispectral image are extracted, and through the iterative closest point, the ICP algorithm is used to perform initial registration of the 2D image data and the 3D point cloud; a multi-scale optimization strategy is adopted to accurately register the pixel-level local details and align the crop structure at the overall scale; then the collected thermal imaging matrix is normalized to obtain a temperature distribution matrix, weights are assigned to it based on correlation calculation, and finally the weighted fusion algorithm is used to integrate the temperature distribution matrix with the texture feature matrix and the band feature matrix to form a 3D multi-modal map containing color texture, spectral, and temperature information, realizing the fusion of the image and the point cloud.

10. The control method of the device for obtaining multi-phenotypic trait information of facility environment crops according to claim 7, characterized in that It also includes the following steps: The temperature sensor (27) collects the temperature information inside the heat dissipation housing (11) and transmits it to the terminal control module; When the temperature information collected by the temperature sensor (27) is lower than the lower limit value T1, the terminal control module controls the cooling fan (26) to stop working. When the collected temperature is between T1 and T2, the terminal control module starts the cooling fan (26); when the cooling fan (26) is running, if the temperature information collected by the temperature sensor (27) still exceeds the upper limit value T2, the terminal control module controls the imaging module to stop working.

Images