Plant growth state deep learning identification method and system based on plant lamp

Abstract

The invention relates to the technical field of deep learning recognition, and discloses a plant growth state deep learning recognition method and system based on a plant lamp. The method comprises the following steps: carrying out multi-modal data acquisition on a plant growth area by adopting a track type multispectral camera array matched with a fixed LED plant light supplement lamp to obtain a plant image data set and an environment parameter data set; performing feature extraction to obtain a plant growth feature vector; carrying out plant growth state identification and light supplement demand prediction based on a multi-task deep learning model of an improved Transform architecture, and obtaining a plant growth stage discrimination result and light supplement demand spatial and temporal distribution data; and an initial light supplement parameter optimization model is constructed based on the plant growth stage discrimination result and the light supplement demand spatial-temporal distribution data, and an LED plant light supplement lamp illumination parameter sequence is obtained through deep Q network iterative optimization, so that efficient optimization and dynamic adjustment of light supplement parameters are realized, and the light supplement control precision is significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning recognition technology, and in particular to a deep learning recognition method and system for plant growth status based on plant lamps. Background Art

[0002] The lighting parameters of traditional LED plant lighting systems are pre-set and difficult to adjust in real time according to the growth status of plants, resulting in unsatisfactory lighting effects and low energy efficiency. In addition, existing lighting control methods mainly rely on manual experience or simple environmental parameter feedback, lacking accurate identification of plant growth status and accurate prediction of dynamic lighting needs, and unable to meet the differentiated lighting needs of plants at different growth stages.

[0003] In recent years, deep learning technology has made significant breakthroughs in the fields of computer vision and intelligent control, providing new technical means for plant growth status recognition and lighting parameter optimization. However, there are still many challenges in applying deep learning technology to plant lighting control research: the plant growth process involves multiple physiological characteristic parameters, and how to effectively integrate multimodal data and extract key features is still a difficulty; secondly, plant growth has obvious temporal and spatial heterogeneity, and traditional deep learning models are difficult to capture information in these two dimensions at the same time; the optimization of lighting parameters is a complex dynamic decision-making process, and how to build an efficient online learning and optimization mechanism still needs in-depth research. Summary of the invention

[0004] The present invention provides a deep learning recognition method and system for plant growth status based on plant lamps, which are used to achieve efficient optimization and dynamic adjustment of fill light parameters, and significantly improve the accuracy of fill light control.

[0005] In a first aspect, the present invention provides a deep learning recognition method for plant growth status based on plant lamps, and the deep learning recognition method for plant growth status based on plant lamps comprises: A track-mounted multispectral camera array matched with a fixed LED plant fill light is used to collect multimodal data of the plant growth area, and a plant image dataset and an environmental parameter dataset containing track location points are obtained; Performing feature extraction on the plant image dataset and the environmental parameter dataset to obtain a plant growth feature vector; The plant growth feature vector is input into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, thereby obtaining plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; An initial fill light parameter optimization model is constructed based on the plant growth stage discrimination result and the fill light demand spatiotemporal distribution data, and an LED plant fill light illumination parameter sequence is obtained through deep Q network iterative optimization.

[0006] In a second aspect, the present invention provides a deep learning recognition system for plant growth status based on plant lamps, and the deep learning recognition system for plant growth status based on plant lamps includes: The acquisition module is used to collect multimodal data of the plant growth area using a track-mounted multispectral camera array matched with a fixed LED plant fill light, and obtain a plant image data set and an environmental parameter data set containing track position points; A feature extraction module, used to extract features from the plant image dataset and the environmental parameter dataset to obtain a plant growth feature vector; A prediction module, which is used to input the plant growth feature vector into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, and obtain plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; The iterative module is used to construct an initial fill light parameter optimization model based on the plant growth stage judgment result and the fill light demand spatiotemporal distribution data, and obtain the LED plant fill light illumination parameter sequence through deep Q network iterative optimization.

[0007] The third aspect of the present invention provides a deep learning recognition device for plant growth status based on plant lights, comprising: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory so that the deep learning recognition device for plant growth status based on plant lights executes the above-mentioned deep learning recognition method for plant growth status based on plant lights.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned deep learning identification method of plant growth status based on plant lights.

[0009] In the technical solution provided by the present invention, a multi-spectral camera array is installed on the track-type LED fill light, combined with a variety of environmental sensors, to achieve multi-dimensional and all-round data collection of the plant growth area, and an improved image processing and feature extraction algorithm is used to accurately extract features such as plant leaf morphology, temperature distribution, photosynthesis, etc., and a comprehensive plant growth feature representation is obtained through feature fusion. The multi-task deep learning model based on the improved Transformer architecture realizes the accurate identification of plant growth status and accurate prediction of fill light demand, and the model has good spatiotemporal feature capture capabilities. The fill light parameter optimization model is constructed by a deep Q network, and the experience playback mechanism is combined to achieve efficient optimization and dynamic adjustment of the fill light parameters, significantly improving the accuracy of the fill light control. The industrial bus communication and multi-channel drive control scheme are adopted to achieve the precise execution of the fill light parameters, ensuring the reliable transmission and execution of the control instructions. The fill light effect is evaluated in real time by a deep learning evaluation model, and the optimization model is updated online based on the evaluation results, ensuring the continuous optimization of the fill light control strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 A schematic diagram of the process of a deep learning method for identifying plant growth status based on plant lamps provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of a deep learning recognition system for plant growth status based on plant lamps provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a deep learning recognition device for plant growth status based on plant lights provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change based on actual conditions.

[0014] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0015] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0016] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0017] See also Figure 1 , Figure 1 A schematic diagram of a process flow of a deep learning method for identifying plant growth status based on plant lamps provided in an embodiment of the present application, such as Figure 1 As shown, the deep learning identification method of plant growth status based on plant lights provided in the embodiment of the present application includes steps S100 to S600.

[0018] Step S100, using a track-type multispectral camera array matched with a fixed LED plant fill light to collect multimodal data of the plant growth area, and obtaining a plant image data set and an environmental parameter data set containing track position points; It is understandable that the execution subject of the present invention may be a plant growth status deep learning recognition system based on plant lights, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0019] Specifically, the moving track of the track-type multispectral camera array is electronically mapped, and the mapping data of the track coordinates is constructed through a precise calibration process. In this process, the geometric information of the track, the device movement path, and the spatial reference point are recorded to ensure the positioning accuracy and repeatability of subsequent data acquisition. The acquisition position of the multispectral camera array is divided based on the track coordinate mapping data, and N track position points are determined according to the preset track step and coverage range. Each track position point corresponds to a clear coordinate to guide the specific acquisition position of the multispectral camera in space. In the multispectral data acquisition process, for each track position point, the system's multispectral data acquisition trigger control mechanism is used to orderly trigger different types of cameras for synchronous acquisition. The visible light camera is responsible for collecting the apparent characteristics of the plant, which include leaf color, morphology, texture and other information; the infrared camera is used to collect the temperature distribution characteristics of the plant, and reflects its thermal condition and transpiration state by recording the infrared radiation intensity on the plant surface; the chlorophyll fluorescence camera specializes in collecting photosynthesis-related characteristics of the plant, and reveals the photosynthesis efficiency and light response state of the plant by detecting chlorophyll fluorescence signals. The image data collected by these different types of cameras constitute the initial multimodal image data set. The initial data is subjected to image denoising, and an image denoising algorithm, such as Gaussian filtering, median filtering, etc., is used to remove random noise in the image. At the same time, considering that the multispectral data comes from different types of cameras, their spatial resolution and field of view are different, so the data is spatially registered. By matching feature points and applying affine transformation or perspective transformation, different types of images are spatially aligned to generate a high-quality plant image data set containing track position points. In terms of environmental parameter data collection, for N track position points, multiple environmental data are collected by sensors arranged on the track device. The temperature and humidity sensor monitors the environmental temperature and humidity data of each position point in real time, and records the changes in water vapor content and temperature in the air; the carbon dioxide sensor is used to collect carbon dioxide concentration data in the environment, which is helpful for analyzing plant photosynthesis and respiration; the light intensity sensor collects environmental light data and records the light intensity distribution in the plant growth area. The original environmental parameter data is detected for outliers, and the abnormal data that does not conform to the actual situation is identified and eliminated through statistical methods or threshold-based detection mechanisms. The data were standardized to ensure that different parameters had a uniform scale and an environmental parameter data set was obtained.

[0020] Step S200, extracting features from the plant image data set and the environmental parameter data set to obtain a plant growth feature vector; Specifically, the visible light images in the plant image dataset are preprocessed by image segmentation, and the plant images are segmented at the pixel level using a deep learning segmentation algorithm (such as U-Net or Mask R-CNN) to generate an image feature map array containing the main structural features of the plant. Based on the image feature map array, the leaf area of ​​the plant is focused on, the leaf area is extracted by the segmentation algorithm, and the mask data of the leaf area is generated. The mask data of the plant leaf area is subjected to morphological processing and edge detection. The noise is removed and the boundary details are enhanced by applying morphological algorithms such as dilation, erosion, and opening and closing operations. The edge contour information of the plant leaves is extracted by combining Canny edge detection or Sobel operator to generate a set of contour feature points. Based on the set of contour feature points, the key morphological parameters of the plant are calculated, including the area, number, plant height, and crown width of the leaf. The calculation of the leaf area is achieved by multiplying the number of pixels of the mask data by the spatial resolution, and the number of leaves is counted by the connected domain analysis algorithm; the plant height and crown width need to be measured by identifying the upper and lower boundaries and the left and right boundaries of the plant in combination with the scale information calibrated in the image. These calculation results constitute the plant morphological feature data, which reflects the external geometric morphological information of the plant. At the same time, for the infrared image data in the plant image data set, it is necessary to extract and analyze the temperature distribution characteristics of the plant. Through the infrared imaging data, a temperature distribution feature mapping matrix is ​​generated, and each pixel in the mapping matrix represents the temperature value distribution on the plant surface. The temperature distribution feature mapping matrix is ​​statistically analyzed, such as calculating the mean, maximum, minimum and standard deviation of the temperature, to extract feature data reflecting the overall and local thermal distribution of the plant, revealing the transpiration, environmental adaptability and potential thermal stress of the plant. For the chlorophyll fluorescence images obtained in the plant image data set, feature extraction is performed for its photosynthesis characteristics. By analyzing the chlorophyll fluorescence signal, a photosynthesis feature mapping matrix is ​​generated, and each pixel in the matrix represents the photosynthesis efficiency of plant leaves in different spatial regions. Combined with the spatial feature analysis of photosynthesis, the photosynthesis efficiency difference data of different parts of the plant are extracted, such as analyzing the light response characteristics of the top and bottom of the leaf and the distribution pattern of the fluorescence signal strength. These feature data reveal the photosynthetic capacity of the plant and its adaptation to the light environment, and constitute the photosynthetic feature data of the plant. Time series feature extraction is performed on the temperature data, humidity data, carbon dioxide concentration data and illuminance data in the environmental parameter data set. Using time series analysis methods (such as short-time Fourier transform or long short-term memory network, LSTM), periodic, trend and mutation characteristics are extracted from these time series data. For example, the day-night variation of light intensity, the long-term fluctuation pattern of temperature and humidity, and the sudden increase of carbon dioxide concentration can be identified. The extracted environmental feature data can help understand the dynamic response of plants in a specific growth environment.The plant morphological feature data, plant temperature distribution feature data, plant photosynthetic feature data and environmental feature data are feature encoded and fused to generate a plant growth feature vector. In the fusion process, the feature embedding method is used to map data of different dimensions into the same vector space, and weighted coding, attention mechanism or other feature fusion algorithms are used to ensure that the information association between multimodal data is fully reflected.

[0021] Step S300, inputting the plant growth feature vector into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, and obtaining plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; Specifically, the plant growth feature vector is input into a multi-head attention encoder in a multi-task deep learning model based on an improved Transformer architecture for parallel processing. The multi-head attention encoder contains 8 attention heads, each of which uses a 64-dimensional query vector, a key vector, and a value vector, which are used to calculate feature weights through a scaled dot product attention mechanism. Through this mechanism, the model can identify important patterns in the input feature vector and dynamically adjust the weights. At the same time, the sequential information of the input feature vector is explicitly embedded into the encoder in combination with sinusoidal position encoding to generate an initial feature representation matrix. The initial feature representation matrix is ​​input into a three-layer cascaded Transformer encoding block in the multi-task deep learning model for feature enhancement. Each layer of the Transformer encoding block consists of a 512-dimensional multi-head self-attention layer, a 2048-dimensional feedforward neural network layer, and a LayerNorm normalization layer. In the multi-head self-attention layer, the model captures the global dependencies between features by calculating multiple attention mechanisms in parallel, while the feedforward neural network layer extracts and processes feature patterns through nonlinear activation functions to enhance the representation ability of features. The LayerNorm normalization layer is used to normalize the output of each layer to ensure numerical stability during training. After three layers of cascade processing, the features are further extracted and integrated to obtain an enhanced feature matrix, which contains deep information in the plant growth feature vector. The enhanced feature matrix is ​​input into the feature diversion module in the multi-task deep learning model for task separation. The feature diversion module accurately divides the features and sends the task-related information to the growth state recognition branch and the supplementary light demand prediction branch to form a dual-task feature representation. In the growth state recognition branch, the growth state features in the dual-task feature representation are input into a fully connected neural network, which contains 4 hidden layers with hidden layer dimensions of 512, 256, 128 and 64 respectively. Each layer uses the ReLU activation function for nonlinear mapping to extract the deep features of the plant growth state. The probability score of the growth stage is calculated through the output layer to generate a multi-classification probability distribution. The probability is normalized using the Softmax function to obtain the discrimination result of the plant growth stage, which accurately describes the current growth stage of the plant. At the same time, the branch features of fill light demand prediction in the dual-task feature representation are input into the spatiotemporal attention module, which combines the spatial attention map and the temporal attention vector to capture the dynamic change pattern of fill light demand in the spatial and temporal dimensions. The spatial attention map dynamically allocates attention weights by focusing on specific spatial areas, such as the key parts of the plant growth state, while the temporal attention vector captures the temporal change trend of the lighting demand by analyzing the features of different time steps. The spatiotemporal attention module adopts a gated fusion mechanism to selectively fuse spatial and temporal features through a gating function to generate a fill light demand feature map.Perform feature extraction on the fill light demand feature map to extract key fill light demand information and generate a fill light demand prediction tensor. Through the reconstruction of this tensor, it is converted into a data format with a clear spatiotemporal distribution structure to obtain the spatiotemporal distribution data of the fill light demand.

[0022] Step S400: construct an initial fill light parameter optimization model based on the plant growth stage discrimination result and the fill light demand spatiotemporal distribution data, and obtain the LED plant fill light illumination parameter sequence through deep Q network iterative optimization.

[0023] Specifically, the plant growth stage discrimination results and the spatiotemporal distribution data of the fill light demand are input into the state encoder, and the state encoder is used to encode the input information into a high-dimensional fill light environment state vector. The fill light environment state vector is feature-divided to construct an action space. The action space includes three key dimensions: light intensity, spectral ratio and illumination angle. Each dimension directly affects the fill light effect and energy efficiency. In order to facilitate model processing, this multi-dimensional continuous action space is discretized to generate a discrete action matrix. Each row of the action matrix represents a possible fill light parameter combination, forming a complete fill light parameter action set. Through this discretized action set, the fill light problem is converted into a processable reinforcement learning task. Based on the fill light environment state vector and the fill light parameter action set, a deep Q network is constructed to optimize the fill light strategy. The deep Q network adopts a dual-stream architecture, in which one branch calculates the state value function of the fill light action and the other branch calculates the advantage function of the specific action. The two are combined to evaluate the Q value of each action to form an initial fill light parameter optimization model. At this stage, the network finds the optimal parameter selection under different fill light environments through the mechanism of reinforcement learning. In the specific optimization process, for the fill light environment state vector of each track position point, the deep Q network calculates the Q value of different fill light parameter actions. The Q value represents the total benefit obtained after selecting a fill light parameter action under the current state. In order to strike a balance between exploring new strategies and using existing strategies, the ε-greedy strategy is used to select the optimal action. This strategy selects random actions for exploration with a certain probability, and selects the action with the highest current Q value in other cases. In this way, the initial fill light parameter combination of each track position point is obtained. The initial fill light parameter combination is stored in the experience replay buffer pool. Through the experience replay mechanism, batches of data are randomly sampled from the buffer pool regularly for training the deep Q network. During training, the temporal difference algorithm is used to calculate the target Q value. The weight parameters of the network are updated by calculating the gradient, so that the model can gradually learn the ability to optimize the fill light parameters. The updated deep Q network outputs the target fill light parameter combination. Since the action parameters between track position points have discontinuity problems caused by discretization, these parameters are smoothed. The bilinear interpolation method is used to make the parameters between the track position points continuous in time and space, generate a smooth fill light parameter curve, and ensure the continuity and stability of the lighting conditions during the fill light process. The continuous fill light parameter curve is reorganized according to the track position points, and the light intensity, spectral ratio, and illumination angle are normalized to ensure that the parameters are within the same numerical range to avoid the impact of scale differences on subsequent applications. Finally, the LED plant fill light illumination parameter sequence is obtained. This sequence can change with time and track position, and dynamically adjust the fill light strategy to meet the real-time needs of plant growth, while maximizing energy efficiency and plant growth benefits.

[0024] The LED plant fill light illumination parameter sequence is converted to a protocol, and the control parameters of the illumination intensity, spectral ratio and illumination angle are sent to the LED fill light controller through the industrial bus to generate the corresponding fill light control instruction. In the fill light control instruction, the illumination intensity parameter is converted into a PWM duty cycle modulation signal. This signal adjusts the output power of LEDs of different bands through the four constant current sources in the LED drive circuit to achieve precise control of the target illumination intensity. In order to achieve dynamic adjustment of different spectra, the driving current ratio of the red light, blue light and far-red light LEDs is calculated according to the spectral ratio parameters in the fill light control instruction. The current value of each band LED is accurately adjusted through the multi-channel LED drive circuit, so that the LED light source outputs according to the target spectral ratio, and the plant responds to specific spectral requirements. The adjustment of the illumination angle is based on the pitch angle parameters in the fill light control instruction. After parsing these parameters, the stepper motor control system drives the pitch angle adjustment mechanism of the LED fill light. The stepper motor has high control accuracy, which can ensure that the LED fill light can adjust the target illumination angle and provide directional fill light for different plant areas. This motion trajectory can cover every track position point in the plant growth area to achieve comprehensive fill light, and optimize energy efficiency through fill light time control at the track position point. In order to ensure real-time monitoring and dynamic adjustment of the fill light system, the electrical parameters of the LED fill light are collected using voltage sensors, current sensors and temperature sensors, and the spectrum output data of the LED fill light is collected in combination with the spectrum sensor to construct the real-time working status data of the LED fill light. Combined with the plant growth status indicators, the real-time working status data and plant growth indicators are evaluated through deep learning technology to calculate the fill light effect score. The evaluation model is based on the matching degree between the plant growth characteristic data and the actual light output, combined with the change trend of the growth state, to comprehensively generate the fill light effect score. If the fill light effect score is lower than the preset threshold, it means that the current fill light parameters fail to meet the optimal requirements of plant growth. At this time, the initial fill light parameter optimization model is updated online. In the online update process, the collected real-time data is used as a new input sample, and the parameters of the deep Q network are dynamically adjusted to enable the model to re-optimize the fill light strategy and generate a target fill light parameter optimization model that better meets actual needs.

[0025] The real-time working status data is grouped and processed according to the track position points to ensure that each group of data can reflect the fill light status and environmental conditions at a specific position point. The grouped data includes multiple parameters such as light intensity, spectral ratio, illumination angle, voltage, current and temperature. These data are standardized to eliminate the differences in dimensions and scales, and the fill light status feature sequence is obtained to describe the operating status of the fill light at each track position point. Based on the fill light status feature sequence, the fill light execution feature vector is constructed. The fill light execution feature vector is a high-dimensional representation of the actual execution result of the fill light system, which can capture the correlation between the fill light status and plant growth. At the same time, in order to quantify the plant growth effect, the plant growth indicators are comprehensively evaluated. By analyzing the three key indicators of plant growth rate change rate, chlorophyll content change rate and photosynthesis efficiency change rate, and performing weighted calculations on them, a comprehensive evaluation index for plant growth is generated. In the weighted calculation process, the weights of different indicators are adjusted according to actual needs. For example, photosynthesis efficiency has a higher weight in some application scenarios to highlight its impact on plant health. The fill light effect score is calculated based on the fill light execution feature vector and the comprehensive evaluation index for plant growth. The fill-light effect score reflects the degree of influence of the fill-light parameters on plant growth in actual implementation, and by comparing this score with the preset score threshold, it is determined whether the current fill-light parameters need to be adjusted. When the fill-light effect score is lower than the preset threshold, it means that the existing fill-light strategy has failed to achieve the expected effect. At this time, the initial fill-light parameter optimization model is updated online. The first step of the online update is to calculate the gradient value of the fill-light effect score relative to the parameters of the initial fill-light parameter optimization model. The calculation of the gradient value is based on the back propagation process of the loss function, which accurately reflects the direction and amplitude of the current parameter adjustment. The calculated parameter update gradient is optimized by the Adam optimizer. The Adam optimizer is an optimization algorithm with an adaptive learning rate that can effectively combine the first-order momentum and second-order momentum information to quickly find the parameter optimization direction. Under the processing of the Adam optimizer, the parameter optimization direction is generated, which is used to guide the adjustment of the deep Q network weights of the initial fill-light parameter optimization model. As the core model of fill-light parameter optimization, the weight update of the deep Q network can directly affect the model's ability to evaluate the value of the fill-light environment state, thereby adjusting the fill-light strategy. After the deep Q network weights are optimized, the updated model parameters are reloaded into the deep Q network, and the parameters of its value evaluation network are updated. The value evaluation network is a key component of the deep Q network. By learning the value relationship between the environmental state and the fill light parameter action, it provides an accurate reference for the fill light decision at each track position. After online update, the value evaluation network has higher decision accuracy and environmental adaptability, and finally generates the target fill light parameter optimization model.

[0026] In the embodiment of the present invention, by equipping the track-type LED fill light with a multi-spectral camera array and combining a variety of environmental sensors, multi-dimensional and all-round data collection of the plant growth area is achieved, and the improved image processing and feature extraction algorithms are used to accurately extract the features of plant leaf morphology, temperature distribution, photosynthesis, etc., and a comprehensive plant growth feature representation is obtained through feature fusion. The multi-task deep learning model based on the improved Transformer architecture realizes the accurate identification of plant growth status and accurate prediction of fill light demand, and the model has good spatiotemporal feature capture capabilities. The fill light parameter optimization model is constructed by a deep Q network, and the experience playback mechanism is combined to achieve efficient optimization and dynamic adjustment of fill light parameters, significantly improving the accuracy of fill light control. The industrial bus communication and multi-channel drive control scheme are adopted to achieve accurate execution of fill light parameters and ensure reliable transmission and execution of control instructions. The fill light effect is evaluated in real time by a deep learning evaluation model, and the optimization model is updated online based on the evaluation results, ensuring the continuous optimization of the fill light control strategy.

[0027] In a specific embodiment, the process of executing step S100 may specifically include the following steps: Electronically mapping the moving track of the track-type multispectral camera array to obtain track coordinate mapping data, and dividing the acquisition position of the multispectral camera array based on the track coordinate mapping data to obtain N track position points; Perform multispectral data acquisition trigger control on N track position points, collect plant appearance characteristics through visible light camera, collect plant temperature distribution characteristics through infrared camera, collect plant photosynthetic characteristics through chlorophyll fluorescence camera, obtain original multimodal image data, and perform image denoising and spatial registration on the original multimodal image data to obtain a plant image dataset containing track position points; The environmental data of N track positions are collected, the environmental temperature and humidity data are collected through the temperature and humidity sensors, the carbon dioxide concentration data are collected through the carbon dioxide sensor, and the environmental light data are collected through the illuminance sensor to obtain the original environmental parameter data. The original environmental parameter data is then subjected to outlier detection and data standardization to obtain the environmental parameter data set.

[0028] Specifically, the moving track of the track-mounted multispectral camera array is electronically mapped. The spatial coordinates of each acquisition position on the track are measured and recorded. By arranging multiple reference points with known positions on the track, such as using a high-precision laser rangefinder or total station, the coordinates of key positions of the track are collected to establish the global coordinate system of the track. These coordinate data are expressed as ,in Track points Position in three-dimensional space. The discrete points are fitted by curve fitting or interpolation algorithms (such as spline interpolation) to generate a continuous path for the entire track. The electronic map of the track is represented as a function ,in The track is divided into two parts: uniform or adaptive track position points are used to guide the acquisition task of the multispectral camera array. For example, if the total track length is , if you want to divide it evenly into location points, then the interval between each location point is The coordinates of the location point are expressed as ,in For the For multispectral data acquisition, a multispectral camera array is used to accurately image plants. At each orbital position, the visible light camera, infrared camera, and chlorophyll fluorescence camera are synchronously controlled by a trigger control system to collect the plant's appearance, temperature distribution, and photosynthesis characteristics. The visible light camera is used to capture the color, morphology, and texture characteristics of the plant, and its imaging data is represented in matrix form. ,in and are the height and width of the image, is the number of color channels. The infrared camera collects the thermal radiation data of the plant and obtains the temperature distribution matrix through thermal imaging. , where each pixel represents the temperature of the plant surface. The chlorophyll fluorescence camera is used to detect the fluorescence signal related to plant photosynthesis, and its output is a fluorescence intensity matrix Since the collected multimodal image data is interfered by noise, the original data is subjected to noise reduction processing. The Gaussian filtering algorithm is used, and the formula is: ; in is the original image, is the pixel window, is the standard deviation of the Gaussian kernel. This method is used to smooth the image and remove random noise. Due to the different viewing angles and resolutions of the multispectral camera array, spatial registration is performed and the transformation matrix between cameras is calculated by feature point matching (such as SIFT algorithm). , and use an affine transformation to align the images: ; in is the pixel coordinate of the target image, is the pixel coordinate of the original image. After completing the image data processing, collect environmental data. The temperature and humidity data of the track position point are collected through the temperature and humidity sensor, and its output is the temperature sequence and humidity series The carbon dioxide concentration data recorded by the carbon dioxide sensor is expressed as , and the light intensity recorded by the light sensor is . Perform outlier detection on the original environmental parameter data and use the triple standard deviation method to define the outlier range: ; in is the parameter mean, is the parameter standard deviation. For data points that are out of range, they are replaced with the mean or interpolated values. To unify the data scale, all environmental parameters are standardized, and the formula is: ; in is the original data value, is the mean, is the standard deviation. The standardized data constitutes the environmental parameter data set , combined with the plant image dataset, it provides high-quality multimodal data input for subsequent analysis.

[0029] In a specific embodiment, the process of executing step S200 may specifically include the following steps: Perform image segmentation preprocessing on the visible light images in the plant image data set to obtain an image feature mapping array, and perform plant leaf region segmentation on the image feature mapping array to obtain plant leaf region mask data; Perform morphological processing and edge detection on the mask data of plant leaf regions to obtain a set of contour feature points, and based on the set of contour feature points, calculate the plant morphological parameters of leaf area, leaf number, plant height, and crown width to obtain plant morphological feature data; Calculate the temperature distribution characteristics of the infrared image data in the plant image data set to obtain a temperature distribution feature mapping matrix, and perform statistical analysis on the temperature distribution feature mapping matrix to obtain plant temperature distribution feature data; Extract photosynthesis features from chlorophyll fluorescence images in plant image datasets to obtain photosynthesis feature mapping matrix, and perform spatial feature analysis on the photosynthesis feature mapping matrix to obtain plant photosynthesis feature data; Extract time series features of temperature data, humidity data, carbon dioxide concentration data and illuminance data in the environmental parameter data set to obtain environmental feature data; The plant morphological feature data, plant temperature distribution feature data, plant photosynthetic feature data and environmental feature data are fused by feature coding to obtain a plant growth feature vector.

[0030] Specifically, the visible light images in the plant image dataset are preprocessed by image segmentation. By using a deep learning segmentation algorithm (such as U-Net or DeepLab), the original visible light images are Divided into different regions, and are the height and width of the image, is the number of color channels. The goal of segmentation is to extract the main part of the plant while excluding the background and irrelevant areas. The result of image segmentation is represented as an image feature map array , where the value of each pixel represents the category it belongs to (such as leaf, stem or background). The image feature map array is processed by region growing or connected domain analysis algorithm, focusing on plant leaf region segmentation and generating plant leaf region mask data ,in Represents pixels Belongs to the leaf area, The mask data of the plant leaf area is morphologically processed to remove isolated noise points through dilation and erosion operations, and to fill small holes in the leaf area. The mask data after morphological processing is used for edge detection, and the boundary points of the leaf are extracted using the Sobel or Canny operator to generate a set of contour feature points. ,in For the The coordinates of the contour feature points, is the total number of boundary points. Based on these contour feature points, the geometric parameters of the blade are calculated. For example, the blade area It is obtained by calculating the total number of leaf pixels in the mask data and multiplying it by the area of ​​a single pixel. The formula is: ; in, is the area of ​​a single pixel in the actual scene. The number of leaves is calculated by connecting domain analysis to count the leaf area. The crown width is calculated by calculating the vertical distance from the top to the bottom of the plant. is the horizontal distance between the left and right boundaries of the plant. These parameters are combined to form the morphological characteristic data of the plant. The temperature distribution characteristics of the infrared image data in the plant image dataset are calculated. Infrared image Each pixel value in represents the temperature of the plant surface. The key features of temperature distribution, including the average temperature, are extracted through statistical analysis. , Maximum temperature , minimum temperature and standard deviation The specific calculation formula is as follows: ; ; These characteristic data reflect the thermal distribution state of plants, reveal their transpiration and potential heat stress, and constitute the temperature distribution characteristic data of plants. For chlorophyll fluorescence images in plant image datasets, a specific photosynthesis parameter extraction technique is used to generate photosynthesis feature mapping matrices. The intensity of the fluorescence signal reflects the activity of plant photosynthesis. Commonly used fluorescence parameters include the maximum fluorescence value , minimum fluorescence value and photochemical efficiency The calculation formula for photochemical efficiency is: ; in and Represent the maximum and minimum values ​​of the fluorescence signal respectively. By performing spatial feature analysis on the photosynthesis feature mapping matrix, the distribution pattern of plant photosynthesis efficiency is identified and key features are extracted as plant photosynthetic feature data. At the same time, time series feature extraction is performed on the temperature, humidity, carbon dioxide concentration and light intensity data in the environmental parameter data set. Assume that the time series of these parameters are ,in is the number of time steps. The frequency domain features are extracted by short-time Fourier transform, and the moving average and change rate are calculated using a sliding window to obtain feature data reflecting the dynamic changes of the environment. The plant morphological feature data, plant temperature distribution feature data, plant photosynthetic feature data and environmental feature data extracted above are fused by feature coding to generate a plant growth feature vector ,in is the dimension of the fused feature vector. The fusion method uses feature concatenation, attention mechanism or embedding layer of deep learning model to ensure the correlation of multimodal features.

[0031] In a specific embodiment, the process of executing step S300 may specifically include the following steps: The plant growth feature vector is input into the multi-head attention encoder in the multi-task deep learning model based on the improved Transformer architecture for parallel processing. The multi-head attention encoder contains 8 attention heads, each of which uses a 64-dimensional query vector, key vector, and value vector. The feature weight is calculated through the scaled dot product attention mechanism and combined with the sinusoidal position encoding to obtain the initial feature representation matrix. The initial feature representation matrix is ​​input into the three-layer serialized Transformer encoding block in the multi-task deep learning model. Each Transformer encoding block contains a 512-dimensional multi-head self-attention layer, a 2048-dimensional feedforward neural network layer, and a LayerNorm normalization layer to obtain an enhanced feature matrix. The enhanced feature matrix is ​​input into the feature diversion module in the multi-task deep learning model for task separation, and a dual-task feature representation is obtained through the growth state recognition branch and the supplementary light demand prediction branch. The growth state recognition branch features in the dual-task feature representation are input into the fully connected neural network in the multi-task deep learning model. The fully connected neural network contains 4 hidden layers with hidden layer dimensions of 512, 256, 128, and 64 respectively. The growth state feature vector is obtained, and the probability score of the growth stage is calculated based on the growth state feature vector to obtain the plant growth stage discrimination result. The fill light demand prediction branch features in the dual-task feature representation are input into the spatiotemporal attention module in the multi-task deep learning model. By calculating the spatial attention map and the temporal attention vector, the gated fusion mechanism is used to integrate the spatiotemporal dimension information to obtain the fill light demand feature map. Feature extraction is performed on the fill light demand feature map to obtain a fill light demand prediction tensor, and the fill light demand prediction tensor is reconstructed into a spatiotemporal distribution structure to obtain the fill light demand spatiotemporal distribution data.

[0032] Specifically, the plant growth feature vector Expressed as having features, where each feature has a dimension of Will Input multi-head attention encoder for parallel processing. The multi-head attention encoder consists of 8 attention heads, each of which independently calculates the query vector , key vector Sum value vector These vectors are generated from the input feature vector through a linear transformation, the transformation formula is: ; in is the weight matrix of query, key and value vectors. The feature weight is calculated by the scaled dot product attention mechanism, and the formula is: ; in is a scaling factor to prevent the softmax gradient from disappearing due to excessive dot product values. is the dimension of the key vector. Multi-head attention is achieved by concatenating the outputs of the eight attention heads and linearly transforming them again to obtain the initial feature representation matrix In order to capture the sequential information of the input features, a sinusoidal positional encoding is added to In , the position encoding formula is: ; Where pos is the position index, is the feature dimension index. The initial feature representation matrix Input three layers of cascaded Transformer encoding blocks, each layer contains a 512-dimensional multi-head self-attention layer, a 2048-dimensional feedforward neural network layer and a LayerNorm normalization layer. In each layer, the global feature relationship is captured by the multi-head self-attention layer, and then the nonlinear features are extracted by the feedforward neural network. Finally, LayerNorm is applied to normalize the output of each layer to ensure the numerical stability of the training process. After three layers of processing, the enhanced feature matrix is ​​obtained. . The feature matrix will be enhanced Input feature diversion module to separate tasks into growth state recognition branch features And the characteristics of the fill light demand prediction branch The growth state recognition branch features are input into a fully connected neural network. The fully connected network contains 4 hidden layers with dimensions of 512, 256, 128 and 64 respectively. The layers are nonlinearly transformed by the ReLU activation function. The network output is the growth state feature vector Finally, the probability score of the growth stage is calculated through the Softmax layer ,in is the number of categories in the growth stage, Defined as: ; in Output score of the network for the corresponding category. Input the spatiotemporal attention module and calculate the spatial attention map and the temporal attention vector ,in are spatial resolution and time steps respectively. Spatial attention is calculated by convolutional neural network, and temporal attention is extracted by LSTM to extract time series features. After combining the two, a gated fusion mechanism is used to generate a fill light requirement feature map. , the gated fusion formula is: ; in is the Sigmoid activation function, is the fusion weight. Fill light requirement feature map After convolutional layer and upsampling, the fill light demand prediction tensor is extracted . Through reconstruction, Converted into fill light demand data with spatiotemporal distribution structure , where the value of each point represents the required intensity of fill light at the corresponding spatial position and time point.

[0033] In a specific embodiment, the process of executing step S400 may specifically include the following steps: The plant growth stage discrimination result and the supplementary light demand distribution data are input into the state encoder for state space encoding to obtain the supplementary light environment state vector; The state vector of the fill light environment is characterized and an action space including light intensity, spectral ratio and illumination angle is constructed. The action space is discretized into an action matrix to obtain a set of fill light parameter actions. A deep Q network is constructed based on the fill light environment state vector and the fill light parameter action set. The deep Q network includes a value evaluation network with a dual-stream architecture to obtain an initial fill light parameter optimization model. For the state vector of the fill light environment within the illumination range of the LED plant fill light, the Q value of different fill light parameter actions is calculated through the deep Q network, and the ε-greedy strategy is used to select the optimal action to obtain the initial fill light parameter combination; The initial fill light parameter combination is input into the experience replay buffer pool, and batch data is randomly sampled from the experience replay buffer pool. The target Q value is calculated by the temporal difference algorithm to obtain the parameter update gradient; The weight parameters of the deep Q network are optimized and updated according to the parameter update gradient to obtain the target fill light parameter combination; The light intensity, spectral ratio and irradiation angle of the target fill light parameter combination are normalized to obtain the illumination parameter sequence of the LED plant fill light.

[0034] Specifically, the plant growth stage discrimination result and the spatiotemporal distribution data of the supplementary lighting requirement are input into the state encoder, and these data are encoded to convert them into a supplementary lighting environment state vector suitable for the deep reinforcement learning model. Assume that the plant growth stage discrimination result is a classification result vector ,in is the number of categories in the growth stage, Each value in represents the probability of the plant being in a certain stage. At the same time, the spatiotemporal distribution data of supplementary lighting requirements is represented as a three-dimensional tensor ,in , and Represent the horizontal and vertical resolutions and time steps of the spatial position respectively. The state encoder encodes these input data through a multi-layer perceptron to generate a fill light environment state vector , the formula is: ; Flatten Flatten a 3D tensor into a 1D vector, and are the weight matrix and bias vector of the state encoder respectively, is the activation function (such as ReLU). According to the track position point, the fill light environment state vector Divide, each position point corresponds to a local state vector ,in Indicates the first Based on the state vector of each position point, an action space is constructed, which includes the light intensity , Spectral ratio (corresponding to the ratio of red light, blue light and far-red light), irradiation angle Through discretization, the value of each parameter is divided into several discrete points. The combination of all parameters constitutes the action matrix ,in is the number of actions, is the action dimension. A deep Q network (DQN) is constructed based on the state vector of the fill light environment and the action matrix. DQN adopts a dual-stream architecture, including the state value stream and action advantage flow , the Q value is calculated by the following formula: ; in For the Actions, For all actions. DQN uses this as the initial fill light parameter optimization model. - Greedy strategy selects actions, i.e. based on probability Randomly select actions with probability Select the action with the largest current Q value to get the initial fill light parameter combination. The initial fill light parameter combination is stored in the experience playback buffer pool. , during the training process, from Randomly sample a batch of data ,in is the reward value, is the next state. Update the target Q value through the temporal difference (TD) algorithm: ; in is a discount factor, indicating the importance of future rewards. Calculate the loss function for: ; The loss function is minimized by the Adam optimizer to obtain the parameter update gradient and optimize the weight parameters of DQN. The optimized target fill light parameters are combined and organized into a continuous curve of the track position points. The bilinear interpolation method is used to smooth the parameters between the track position points. Assume that the parameters of two adjacent points on the track are and , the interpolation parameters are: ; in is the interpolation point, The fill light parameter curve is normalized to scale the values ​​of light intensity, spectral ratio, and irradiation angle to a uniform range, generating a sequence of LED plant fill light illumination parameters.

[0035] In a specific embodiment, executing the deep learning recognition method for plant growth status based on plant lamps also includes the following steps: Perform protocol conversion on the illumination parameter sequence of LED plant fill light, and send the control parameters of light intensity, spectrum ratio and illumination angle to the LED fill light controller through the industrial bus to obtain fill light control instructions; The light intensity parameter in the fill light control instruction is converted into a PWM duty cycle modulation signal, and the output power of LEDs in different bands is controlled by the four constant current sources of the LED drive circuit to obtain the target light intensity; The driving current ratio of the red light, blue light and far-red light LEDs is calculated according to the spectrum ratio parameters in the fill light control instruction, and the current value of each band LED is adjusted through the multi-channel LED driving circuit to obtain the target spectrum ratio; The illumination angle parameters in the fill light control command are parsed, and the pitch angle adjustment mechanism of the LED fill light is driven by the stepper motor control system to obtain the target illumination angle; The electrical parameters of the LED fill light are collected through the voltage sensor, current sensor and temperature sensor, and the spectrum output data of the LED fill light is collected through the spectrum sensor to obtain the real-time working status data of the LED fill light; A deep learning evaluation is performed on the real-time working status data and plant growth indicators to calculate the fill light effect score. When the fill light effect score is lower than the preset threshold, the parameters of the initial fill light parameter optimization model are updated online to obtain the target fill light parameter optimization model.

[0036] Specifically, the light intensity in the LED plant light parameter sequence , Spectral ratio , illumination angle Perform protocol conversion and convert the parameter format into a frame structure that complies with the industrial bus communication standard. Transmit it to the LED fill light controller through the industrial bus and generate fill light control instructions. , the controller converts it into a PWM (pulse width modulation) duty cycle signal. The duty cycle of the PWM signal is proportional to the target light intensity, and its calculation formula is: ; in The PWM signal passes through the four constant current source modules of the LED drive circuit to control the output power of the red, blue, far-red and white LEDs respectively, generating the target light intensity. Spectral matching parameters Used to adjust the driving current ratio of each band LED. Assuming the total current is , then the driving current of each band , and It can be calculated as: ; The driving current of each band is adjusted separately through the multi-channel LED driving circuit to ensure that the spectral output meets the target ratio. After the controller analyzes the parameters, it adjusts the pitch angle of the LED fill light through the stepper motor control system. Calculated according to the following formula: ; in is the single-step angle of the stepper motor, and StepsPerRevolution is the number of steps per revolution of the stepper motor. In order to monitor the operating status of the system in real time, the electrical parameters of the LED fill light are collected using voltage sensors, current sensors, and temperature sensors. , and At the same time, the spectral sensor records real-time spectral output data These data constitute the real-time working status data of the LED fill light. . Real-time working status data and plant growth indicators (such as chlorophyll content, photosynthesis efficiency) to conduct deep learning evaluation and calculate the lighting effect score Online update is based on deep reinforcement learning, and the weight parameters of the deep Q network are optimized by collecting new state-action-reward data. The optimized target fill light parameters are reloaded into the controller, and the updated dynamic fill light parameter curve is generated and sent to the system for execution again, realizing dynamic, closed-loop fill light control.

[0037] In a specific embodiment, the execution step performs deep learning evaluation on the real-time working status data and plant growth indicators, calculates the fill light effect score, and when the fill light effect score is lower than a preset threshold, performs online parameter update on the initial fill light parameter optimization model to obtain the target fill light parameter optimization model. The process may specifically include the following steps: The real-time working status data is grouped to obtain multiple groups of working status data, and the light intensity, spectrum ratio, irradiation angle, voltage and current, and temperature parameters in each group of working status data are standardized to obtain a fill light status feature sequence; Based on the feature sequence of the fill light status, the fill light execution feature vector is constructed. At the same time, the growth rate change rate, chlorophyll content change rate, and photosynthesis efficiency change rate in the plant growth indicators are weighted and calculated to obtain the comprehensive evaluation index of plant growth. The fill-lighting effect score is calculated according to the fill-lighting execution feature vector and the comprehensive evaluation index of plant growth, and the fill-lighting effect score is compared with a preset score threshold. When the fill-lighting effect score is lower than the preset threshold, the gradient value of the fill-lighting effect score relative to the initial fill-lighting parameter optimization model parameter is calculated to obtain the parameter update gradient; The parameter update gradient is processed by the Adam optimizer to obtain the parameter optimization direction, and the deep Q network weights of the initial fill light parameter optimization model are updated online based on the parameter optimization direction to obtain the updated model parameters; The updated model parameters are reloaded into the deep Q network, and the parameters of the value evaluation network of the deep Q network are updated to obtain the target fill light parameter optimization model.

[0038] Specifically, the real-time working status data is grouped according to the track position points, and the multi-dimensional data such as light intensity, spectral ratio, irradiation angle, voltage, current, temperature, etc. from different position points are classified into independent data groups. Assume that the track position point on the track is The real-time working status data corresponding to each location point includes the light intensity , Spectral ratio , illumination angle ,Voltage , Current , and temperature Through grouping operations, these data are organized into multiple groups of work status data ,in Indicates the index of the track position point. For each set of working status data, each parameter is standardized to eliminate the difference in dimension and value range and ensure the stability of subsequent analysis. The standardization formula is as follows: ; in is the original parameter value, is the mean value of the parameter, is the standard deviation, is the standardized parameter value. , Perform standardization processing in sequence to obtain the fill light state feature sequence Based on the fill light state feature sequence, a fill light execution feature vector is constructed. Suppose the fill light execution feature vector is , which contains standardized multidimensional features representing track position points At the same time, the growth rate change rate in the plant growth index , Chlorophyll content change rate , and the rate of change of photosynthesis efficiency Perform weighted calculations to generate comprehensive evaluation indicators for plant growth The calculation formula is: ; in is the weight coefficient, which is used to balance the importance of different growth indicators. and comprehensive evaluation index of plant growth , calculate the fill light effect score The scoring model is implemented through regression analysis or deep learning methods, for example, a linear model: ; in is the model weight, is the bias value. Below the preset score threshold , we need to calculate the gradient value of the fill light effect score relative to the initial fill light parameter optimization model parameters to guide the optimization direction of the model. The calculation of the gradient value is based on the loss function , the squared error loss function is defined as: ; in is the target score. Through the back propagation algorithm, the gradient of the loss function to the model parameters is obtained The Adam optimizer is used to optimize the gradient value. The Adam optimizer combines the first-order and second-order momentum. The formula is: ; ; ; in is the momentum parameter, is the learning rate, is the numerical stability parameter, is the optimized model parameter. The updated model parameters are reloaded into the deep Q network, and the value evaluation part of the network is updated online to ensure that the network estimates the value of the state-action pair more accurately. After multiple iterations of optimization, the target fill light parameter optimization model is obtained.

[0039] See also Figure 2 , Figure 2 A schematic block diagram of the structure of a plant growth status deep learning recognition system 200 based on a plant lamp provided in an embodiment of the present application, such as Figure 2 As shown, a plant growth status deep learning recognition system 200 based on plant lamps includes: The acquisition module 210 is used to use a track-type multi-spectral camera array matched with a fixed LED plant fill light to collect multimodal data of the plant growth area, and obtain a plant image data set and an environmental parameter data set containing track position points; A feature extraction module 220 is used to extract features from the plant image data set and the environmental parameter data set to obtain a plant growth feature vector; Prediction module 230, used to input plant growth feature vectors into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, and obtain plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; The iteration module 240 is used to construct an initial fill light parameter optimization model based on the plant growth stage judgment results and the temporal and spatial distribution data of the fill light demand, and obtain the LED plant fill light illumination parameter sequence through deep Q network iterative optimization.

[0040] Through the collaboration of the above components, a multi-spectral camera array is installed on the track-type LED fill light, combined with a variety of environmental sensors, to achieve multi-dimensional and all-round data collection of plant growth areas, and the improved image processing and feature extraction algorithms are used to accurately extract features such as plant leaf morphology, temperature distribution, and photosynthesis, and obtain comprehensive plant growth feature representation through feature fusion. The multi-task deep learning model based on the improved Transformer architecture realizes the accurate identification of plant growth status and accurate prediction of fill light demand, and the model has good spatiotemporal feature capture capabilities. The fill light parameter optimization model is constructed by deep Q network, combined with the experience playback mechanism, to achieve efficient optimization and dynamic adjustment of fill light parameters, and significantly improve the accuracy of fill light control. The industrial bus communication and multi-channel drive control scheme are adopted to achieve accurate execution of fill light parameters and ensure the reliable transmission and execution of control instructions. The fill light effect is evaluated in real time through the deep learning evaluation model, and the optimization model is updated online based on the evaluation results to ensure the continuous optimization of the fill light control strategy.

[0041] See also Figure 3 , Figure 3 A schematic block diagram of the structure of a deep learning recognition device 300 for plant growth status based on a plant lamp provided in an embodiment of the present application, wherein the deep learning recognition device 300 for plant growth status based on a plant lamp includes a processor 301 and a memory 302, wherein the processor 301 and the memory 302 are connected via a system bus 303, wherein the memory 302 may include a non-volatile storage medium and an internal memory.

[0042] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 301, the processor 301 can execute any of the above-mentioned plant growth status deep learning recognition methods based on plant lamps.

[0043] The processor 301 is used to provide computing and control capabilities to support the operation of the entire plant growth status deep learning recognition device 300 based on plant lights.

[0044] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 301, the processor 301 can execute any of the above-mentioned deep learning and recognition methods of plant growth status based on plant lights.

[0045] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation of the plant growth status deep learning identification device 300 based on plant lamps involved in the present application scheme. The specific plant growth status deep learning identification device 300 based on plant lamps may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0046] It should be understood that the processor 301 may be a central processing unit (CPU), and the processor 301 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0047] It should be noted that technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the plant growth status deep learning recognition device 300 based on plant lamps described above can refer to the corresponding process of the aforementioned plant growth status deep learning recognition method based on plant lamps, and will not be repeated here.

[0048] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement a deep learning recognition method for plant growth status based on plant lights as provided in an embodiment of the present application.

[0049] The computer-readable storage medium may be an internal storage unit of the plant growth state deep learning recognition device 300 based on the plant lamp in the aforementioned embodiment, such as a hard disk or memory of the plant growth state deep learning recognition device 300 based on the plant lamp. The computer-readable storage medium may also be an external storage device of the plant growth state deep learning recognition device 300 based on the plant lamp, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the plant growth state deep learning recognition device 300 based on the plant lamp.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0051] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0052] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep learning recognition method for plant growth status based on plant lamps, characterized in that: include: A track-mounted multispectral camera array matched with a fixed LED plant fill light is used to collect multimodal data of the plant growth area, and a plant image dataset and an environmental parameter dataset containing track location points are obtained; Performing feature extraction on the plant image dataset and the environmental parameter dataset to obtain a plant growth feature vector; The plant growth feature vector is input into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, thereby obtaining plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; An initial fill light parameter optimization model is constructed based on the plant growth stage discrimination result and the fill light demand spatiotemporal distribution data, and an LED plant fill light illumination parameter sequence is obtained through deep Q network iterative optimization.

2. The deep learning recognition method for plant growth status based on plant lamps according to claim 1 is characterized in that: The multi-modal data collection of the plant growth area is performed using a track-type multi-spectral camera array matched with a fixed LED plant fill light to obtain a plant image dataset and an environmental parameter dataset containing track position points, including: Electronically mapping the moving track of the track-type multi-spectral camera array to obtain track coordinate mapping data, and dividing the acquisition position of the multi-spectral camera array based on the track coordinate mapping data to obtain N track position points; Performing multispectral data acquisition trigger control on the N track position points, collecting plant appearance characteristics through a visible light camera, collecting plant temperature distribution characteristics through an infrared camera, and collecting plant photosynthetic characteristics through a chlorophyll fluorescence camera to obtain original multimodal image data, and performing image denoising and spatial registration on the original multimodal image data to obtain a plant image data set containing the track position points; The environmental data of the N track position points are collected, the environmental temperature and humidity data are collected through the temperature and humidity sensor, the carbon dioxide concentration data are collected through the carbon dioxide sensor, and the environmental light data are collected through the illuminance sensor to obtain the original environmental parameter data, and the original environmental parameter data are subjected to outlier detection and data standardization processing to obtain the environmental parameter data set.

3. The deep learning recognition method for plant growth status based on plant lamps according to claim 2 is characterized in that: The step of extracting features from the plant image dataset and the environmental parameter dataset to obtain a plant growth feature vector includes: Performing image segmentation preprocessing on the visible light image in the plant image data set to obtain an image feature mapping array, and performing plant leaf region segmentation on the image feature mapping array to obtain plant leaf region mask data; Performing morphological processing and edge detection on the plant leaf region mask data to obtain a set of contour feature points, and based on the set of contour feature points, calculating plant morphological parameters such as leaf area, leaf number, plant height, and crown width to obtain plant morphological feature data; Calculating the temperature distribution characteristics of the infrared image data in the plant image data set to obtain a temperature distribution characteristic mapping matrix, and performing statistical analysis on the temperature distribution characteristic mapping matrix to obtain plant temperature distribution characteristic data; Extracting photosynthesis features from the chlorophyll fluorescence image in the plant image data set to obtain a photosynthesis feature mapping matrix, and performing spatial feature analysis on the photosynthesis feature mapping matrix to obtain plant photosynthesis feature data; Extracting time series features of the temperature data, humidity data, carbon dioxide concentration data and illuminance data in the environmental parameter data set to obtain environmental feature data; The plant morphological feature data, the plant temperature distribution feature data, the plant photosynthetic feature data and the environmental feature data are fused by feature coding to obtain a plant growth feature vector.

4. The deep learning recognition method for plant growth status based on plant lamps according to claim 3 is characterized in that: The plant growth feature vector is input into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, and obtain plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data, including: Input the plant growth feature vector into a multi-head attention encoder in a multi-task deep learning model based on an improved Transformer architecture for parallel processing, wherein the multi-head attention encoder includes 8 attention heads, each of which uses a 64-dimensional query vector, a key vector, and a value vector, calculates feature weights through a scaled dot product attention mechanism, and combines with sinusoidal position encoding to obtain an initial feature representation matrix; Inputting the initial feature representation matrix into a three-layer series-connected Transformer encoding block in the multi-task deep learning model, each layer of the Transformer encoding block includes a 512-dimensional multi-head self-attention layer, a 2048-dimensional feedforward neural network layer and a LayerNorm normalization layer, to obtain an enhanced feature matrix; Inputting the enhanced feature matrix into the feature diversion module in the multi-task deep learning model to perform task separation, and obtaining a dual-task feature representation through a growth state recognition branch and a fill light demand prediction branch; Inputting the growth state recognition branch features in the dual-task feature representation into the fully connected neural network in the multi-task deep learning model, the fully connected neural network includes 4 hidden layers, and the hidden layer dimensions are 512, 256, 128, and 64, respectively, to obtain a growth state feature vector, and calculating the probability score of the growth stage according to the growth state feature vector to obtain a plant growth stage discrimination result; Inputting the fill light requirement prediction branch features in the dual-task feature representation into the spatiotemporal attention module in the multi-task deep learning model, calculating the spatial attention map and the temporal attention vector, and integrating the spatiotemporal dimension information using a gated fusion mechanism to obtain a fill light requirement feature map; Feature extraction is performed on the fill light demand feature map to obtain a fill light demand prediction tensor, and the fill light demand prediction tensor is reconstructed into a spatiotemporal distribution structure to obtain fill light demand spatiotemporal distribution data.

5. The deep learning recognition method for plant growth status based on plant lamps according to claim 4 is characterized in that: The initial fill light parameter optimization model is constructed based on the plant growth stage discrimination result and the fill light demand spatiotemporal distribution data, and the LED plant fill light illumination parameter sequence is obtained through deep Q network iterative optimization, including: Inputting the plant growth stage discrimination result and the fill light demand distribution data into a state encoder for state space encoding to obtain a fill light environment state vector; Performing feature division on the fill light environment state vector, constructing an action space including light intensity, spectral ratio, and illumination angle, and discretizing the action space into an action matrix to obtain a fill light parameter action set; Based on the fill light environment state vector and the fill light parameter action set, a deep Q network is constructed, wherein the deep Q network includes a value evaluation network with a dual-stream architecture, and an initial fill light parameter optimization model is obtained; For the fill light environment state vector within the illumination range of the LED plant fill light, the Q value of different fill light parameter actions is calculated through the deep Q network, and the ε-greedy strategy is used to select the optimal action to obtain the initial fill light parameter combination; Inputting the initial fill light parameter combination into the experience replay buffer pool, and randomly sampling batch data from the experience replay buffer pool, calculating the target Q value through the temporal difference algorithm, and obtaining the parameter update gradient; Optimizing and updating the weight parameters of the deep Q network according to the parameter update gradient to obtain a target fill light parameter combination; The illumination intensity, spectral ratio and illumination angle of the target fill light parameter combination are normalized to obtain an illumination parameter sequence of the LED plant fill light.

6. The deep learning recognition method for plant growth status based on plant lamps according to claim 5 is characterized in that: The plant growth status deep learning recognition method based on plant lamps also includes: The LED plant fill light illumination parameter sequence is converted into a protocol, and the control parameters of the light intensity, the spectrum ratio and the illumination angle are sent to the LED fill light controller through the industrial bus to obtain a fill light control instruction; The light intensity parameter in the fill light control instruction is converted into a PWM duty cycle modulation signal, and the output power of LEDs of different bands is controlled by four constant current sources of the LED drive circuit to obtain the target light intensity; The driving current ratio of the red light, blue light, and far-red light LEDs is calculated according to the spectrum ratio parameters in the fill light control instruction, and the current value of each band LED is adjusted through a multi-channel LED driving circuit to obtain a target spectrum ratio; Parsing the illumination angle parameter in the fill light control instruction, driving the pitch angle adjustment mechanism of the LED fill light through the stepper motor control system, and obtaining the target illumination angle; The electrical parameters of the LED fill light are collected through the voltage sensor, current sensor and temperature sensor, and the spectrum output data of the LED fill light is collected through the spectrum sensor to obtain the real-time working status data of the LED fill light; A deep learning evaluation is performed on the real-time working status data and plant growth indicators to calculate a fill light effect score. When the fill light effect score is lower than a preset threshold, the parameters of the initial fill light parameter optimization model are updated online to obtain a target fill light parameter optimization model.

7. The deep learning recognition method for plant growth status based on plant lamps according to claim 6 is characterized in that: The deep learning evaluation is performed on the real-time working status data and the plant growth index to calculate the fill light effect score, and when the fill light effect score is lower than a preset threshold, the parameters of the initial fill light parameter optimization model are updated online to obtain a target fill light parameter optimization model, including: The real-time working status data is grouped to obtain multiple groups of working status data, and the light intensity, spectrum ratio, irradiation angle, voltage and current, and temperature parameters in each group of working status data are standardized to obtain a fill light state characteristic sequence; Based on the fill light state feature sequence, a fill light execution feature vector is constructed, and at the same time, a weighted calculation is performed on the growth rate change rate, chlorophyll content change rate, and photosynthesis efficiency change rate in the plant growth index to obtain a comprehensive evaluation index for plant growth; Calculating a fill light effect score according to the fill light execution feature vector and the comprehensive evaluation index of plant growth, and comparing the fill light effect score with a preset score threshold; when the fill light effect score is lower than the preset threshold, calculating a gradient value of the fill light effect score relative to an initial fill light parameter optimization model parameter to obtain a parameter update gradient; Performing Adam optimizer processing on the parameter update gradient to obtain a parameter optimization direction, and performing online updating on the deep Q network weights of the initial fill light parameter optimization model based on the parameter optimization direction to obtain updated model parameters; The updated model parameters are reloaded into the deep Q network, and the parameters of the value evaluation network of the deep Q network are updated to obtain a target fill light parameter optimization model.

8. A deep learning recognition system for plant growth status based on plant lamps, characterized in that: The method for performing deep learning recognition of plant growth status based on plant lamps according to any one of claims 1 to 7 comprises: The acquisition module is used to collect multimodal data of the plant growth area using a track-mounted multispectral camera array matched with a fixed LED plant fill light, and obtain a plant image data set and an environmental parameter data set containing track position points; A feature extraction module, used to extract features from the plant image dataset and the environmental parameter dataset to obtain a plant growth feature vector; A prediction module, which is used to input the plant growth feature vector into a multi-task deep learning model based on an improved Transformer architecture to perform plant growth state recognition and supplementary lighting demand prediction, and obtain plant growth stage discrimination results and supplementary lighting demand spatiotemporal distribution data; The iterative module is used to construct an initial fill light parameter optimization model based on the plant growth stage judgment result and the fill light demand spatiotemporal distribution data, and obtain the LED plant fill light illumination parameter sequence through deep Q network iterative optimization.

9. A deep learning recognition device for plant growth status based on plant lamps, characterized in that: The deep learning recognition device for plant growth status based on plant lamps includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the plant growth status deep learning recognition device based on the plant lamp executes the plant growth status deep learning recognition method based on the plant lamp as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, a deep learning method for identifying plant growth status based on a plant lamp as described in any one of claims 1 to 7 is implemented.

Images