Green plant flowering phase monitoring method, device and system based on deep learning

Abstract

The invention discloses a green plant flowering phase monitoring method, device and system based on deep learning, and the method comprises the steps: achieving the cross-modal dynamic enhancement of visible light and near-infrared images through a learnable convolution kernel, and generating a vegetation active mask to precisely position a vegetation region; a channel attention mechanism is combined with mask weighted fusion spectrum and texture features, and non-vegetation noise interference is inhibited; the method comprises the following steps: dynamically associating climate parameters and image features through cross-modal cross attention, and capturing non-linear influence of environmental change on the flowering phase; finally, by means of a cascade residual classifier and a one-way time sequence gating network, multi-scale features and time sequence rules are fused, and high-precision and high-robustness green plant flowering phase monitoring in a complex environment is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a green plant flowering period monitoring method, device and system based on deep learning. Background Art

[0002] Currently, common methods for monitoring plant flowering periods include manual observation and methods based on traditional sensors or simple image recognition technology. Manual observation requires regular field visits to visually observe plant morphological changes and record flowering information. While this method can obtain data to a certain extent, it is extremely cumbersome, requiring significant manpower and time. Furthermore, data accuracy is difficult to guarantee due to differences in subjective perception and experience among observers.

[0003] With advances in technology, image recognition-based techniques for monitoring flowering periods have emerged. These approaches primarily utilize image processing algorithms (such as color space conversion and edge detection) to extract features of buds or petals, and then classify flowering stages using classifiers such as support vector machines and random forests. However, these methods still have significant flaws: First, they rely on a single visible light band and are unable to quantify the physiological activity of vegetation through spectral information (such as near-infrared reflectance), leading to misjudgments of wilted and healthy areas. Second, they fail to effectively suppress noise interference from non-vegetated areas (such as soil and flower pots), and the feature extraction process introduces a large amount of irrelevant background information, affecting the analysis of green plant flowering period monitoring. Third, they do not integrate environmental parameters to dynamically monitor changes in flowering period, making it difficult to address the delayed effects of environmental mutations on flowering period.

[0004] In summary, existing technologies are not accurate enough in monitoring the flowering period of green plants in complex scenarios. Summary of the Invention

[0005] Embodiments of the present invention provide a green plant flowering period monitoring method, device, and system based on deep learning, which can improve the accuracy of flowering period monitoring in complex scenarios.

[0006] An embodiment of the present invention provides a method for monitoring the flowering period of green plants based on deep learning, comprising: Acquire a current time-series image set of a target green plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the image set, and generate a standardized image set including a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolution kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel; Extracting spectral reflectance features and texture features from the standardized image set, combining them with the vegetation activity mask and performing region-constrained fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; Mapping the climate parameters currently collected by the environmental sensor installed on the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate an environmental perception spatiotemporal decision feature vector; Inputting the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generating a probability distribution of the flowering stage of the target green plant through the cross-layer residual connection of the cascade residual classifier; The flowering stage probability distribution and historical flowering stage data are fused based on a unidirectional temporal gating network to generate the flowering stage monitoring result of the target green plant.

[0007] As an improvement to the above solution, the method of acquiring a current time-series image set of a target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the image set, and generating a standardized image set including a visible light image and a near-infrared band image includes the following sub-steps: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

[0008] As an improvement to the above scheme, the spectral reflectance features and texture features from the standardized image set are extracted, combined with the vegetation activity mask, and regionally constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the regional constrained fusion suppresses noise interference in the non-vegetation area of ​​the target green plant through mask weighting, and includes the following sub-steps: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

[0009] As an improvement to the above scheme, the climate parameters currently collected by the environmental sensor installed on the target green plant are mapped into an environmental feature vector, and the environmental feature vector is interacted with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate an environmentally aware spatiotemporal decision feature vector, including the following sub-steps: Characterize the climate parameters currently collected by the environmental sensor installed on the target green plant to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters; The spatiotemporal correlation feature tensor is linearly transformed to generate a cross-modal query vector, and the multi-dimensional environment feature vector is mapped into a key vector and a value vector respectively; Calculating a similarity matrix between the cross-modal query vector and the key vector to generate an environmentally aware attention weight; The value vector is weightedly aggregated according to the attention weight, and the spatiotemporal decision feature vector is output.

[0010] As an improvement to the above solution, the spatiotemporal decision feature vector is input into a pre-built cascade residual classifier, and the probability distribution of the flowering stage of the target green plant is generated through the cross-layer residual connection of the cascade residual classifier, including the following sub-steps: Inputting the spatiotemporal decision feature vector into the first residual module of a pre-built cascade residual classifier, extracting local features through a convolutional layer, and performing a residual connection between the extracted results and the original input features to generate first-level enhanced features; Inputting the first-level enhanced features into the second residual module in the cascaded residual classifier, generating weight coefficients for each feature channel through a channel attention mechanism, and performing dynamic pooling processing on the feature map to generate second-level enhanced features; The second-level enhanced features are subjected to full connection layer mapping and probability normalization processing to output the flowering stage probability distribution.

[0011] As an improvement to the above solution, the method of fusing the flowering stage probability distribution with historical flowering data based on a unidirectional temporal gating network to generate the flowering period monitoring result of the target green plant includes the following sub-steps: Splicing the current flowering stage probability distribution and historical flowering stage data in chronological order into a time series input sequence; The temporal input sequence is modeled through a unidirectional gated recurrent unit, the forgetting ratio of the historical state is controlled by a reset gate, and the fusion weight of the current input is adjusted through an update gate to generate a temporal fusion feature; Perform classification decisions on the time series fusion features and output the flowering period monitoring results.

[0012] Another embodiment of the present invention provides a green plant flowering period monitoring device based on deep learning, including: An image generation module is configured to acquire a current time-series image set of a target plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the image set, and generate a standardized image set comprising a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target plant using a learnable convolution kernel and generates a vegetation activity mask for the target plant based on the spectral reflectance of the near-infrared channel; A feature extraction module is configured to extract spectral reflectance features and texture features from the standardized image set, combine them with the vegetation activity mask, and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; a vector generation module for mapping the climate parameters currently collected by the environmental sensor installed on the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception; A probability prediction module, configured to input the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generate a probability distribution of the flowering stage of the target green plant through a cross-layer residual connection of the cascade residual classifier; The flowering period analysis module is used to generate the flowering period monitoring results of the target green plant by fusing the flowering stage probability distribution and historical flowering period data based on a unidirectional temporal gating network.

[0013] As an improvement to the above solution, the image generation module is specifically configured to: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

[0014] As an improvement to the above solution, the feature extraction module is specifically used to: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

[0015] Another embodiment of the present invention provides a green plant flowering period monitoring system based on deep learning, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the green plant flowering period monitoring method based on deep learning described in the above-mentioned embodiment of the invention.

[0016] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The embodiment of the present invention obtains a time-series image set of the target green plant through an image acquisition device, performs cross-modal dynamic enhancement processing, uses a learnable convolution kernel to separate the brightness and color components, generates a vegetation activity mask based on the spectral reflectance of the near-infrared channel, and obtains a standardized image set containing visible light and near-infrared band maps, thereby quantifying the physiological activity state of the vegetation; then extracts the spectral reflectance characteristics and texture characteristics of the standardized image set, combines them with the vegetation activity mask, uses the channel attention mechanism to perform regional constraint fusion, suppresses noise interference in non-vegetation areas, and generates a spatiotemporal correlation feature tensor; maps the climate parameters collected by the environmental sensor into an environmental feature vector, interacts with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism, generates an environmentally aware spatiotemporal decision feature vector, and dynamically monitors the flowering period in combination with the environmental parameters; finally, inputs the spatiotemporal decision feature vector into a cascade residual classifier, uses cross-layer residual connections to generate a probability distribution of the flowering stage, and then fuses the historical flowering data through a unidirectional temporal gating network to obtain a flowering period monitoring result. In summary, the embodiments of the present invention significantly improve the accuracy and robustness of green plant flowering period monitoring in complex environments through the collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion and environmental perception decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a green plant flowering period monitoring method based on deep learning provided by one embodiment of the present invention; Figure 2 This is a structural diagram of a green plant flowering period monitoring device based on deep learning provided by one embodiment of the present invention; Figure 3 This is a structural diagram of a green plant flowering period monitoring system based on deep learning provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] See also Figure 1 , is a flow chart of a green plant flowering period monitoring method based on deep learning provided by one embodiment of the present invention. The green plant flowering period monitoring method based on deep learning comprises the following steps: S10, acquiring a current time-series image set of the target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the image set to generate a standardized image set including a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolution kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel; S11, extracting spectral reflectance features and texture features from the standardized image set, combining them with the vegetation activity mask, and performing region-constrained fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; S12, mapping the climate parameters currently collected by the environmental sensor of the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception; S13, inputting the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generating a probability distribution of the flowering stage of the target green plant through the cross-layer residual connection of the cascade residual classifier; S14, fusing the flowering stage probability distribution and historical flowering data based on a unidirectional temporal gating network to generate a flowering period monitoring result of the target green plant.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The embodiment of the present invention obtains a time-series image set of the target green plant through an image acquisition device, performs cross-modal dynamic enhancement processing, uses a learnable convolution kernel to separate the brightness and color components, generates a vegetation activity mask based on the spectral reflectance of the near-infrared channel, and obtains a standardized image set containing visible light and near-infrared band maps, thereby quantifying the physiological activity state of the vegetation; then extracts the spectral reflectance characteristics and texture characteristics of the standardized image set, combines them with the vegetation activity mask, uses the channel attention mechanism to perform regional constraint fusion, suppresses noise interference in non-vegetation areas, and generates a spatiotemporal correlation feature tensor; maps the climate parameters collected by the environmental sensor into an environmental feature vector, interacts with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism, generates an environmentally aware spatiotemporal decision feature vector, and dynamically monitors the flowering period in combination with the environmental parameters; finally, inputs the spatiotemporal decision feature vector into a cascade residual classifier, uses cross-layer residual connections to generate a probability distribution of the flowering stage, and then fuses the historical flowering data through a unidirectional temporal gating network to obtain a flowering period monitoring result. In summary, the embodiments of the present invention significantly improve the accuracy and environmental robustness of green plant flowering period monitoring in complex environments through the collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion and environmental perception decision-making.

[0021] As one example, the method of acquiring a current time-series image set of a target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the image set, and generating a standardized image set including a visible light image and a near-infrared band image includes the following sub-steps: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

[0022] In this embodiment, a series of innovative operations are used to generate standardized image sets, focusing on image acquisition and processing, with the goal of improving the quality and feature accuracy of green plant images. First, an image acquisition device is used to obtain the current time-series image set of the target green plant. Considering the spatial misalignment between the visible light band image and the near-infrared band image, spatial registration is performed to achieve pixel alignment and generate a dual-channel image set, laying the foundation for subsequent processing. Next, a separable convolutional network is used to input the dual-channel image set. The first convolution branch extracts brightness component features, and the second convolution branch extracts color component features, effectively decoupling brightness and color to obtain an intermediate feature map. An initial vegetation activity mask is then generated based on the difference in spectral reflectance between the near-infrared band images. Morphological closing operations are performed to eliminate holes and discontinuous edges in the mask, generating a vegetation activity mask with continuous edges and accurately segmenting the vegetation area. Finally, a channel-weighted fusion is performed between the intermediate feature map, which decouples brightness and color, and the vegetation activity mask to highlight the image features of the vegetation area and suppress background noise interference. The output is a standardized image set containing both the visible light image and the near-infrared band image. In summary, this embodiment effectively improves the clarity and accuracy of green plant features in images, provides high-quality data support for subsequent image-based green plant flowering period monitoring, and enhances the reliability and stability of monitoring results.

[0023] The specific working process of this embodiment is as follows: Step 1: Generate a dual-channel image set through spatial registration: By designing a spatial registration algorithm based on deformable convolution, the visible light band image (RGB) and the near-infrared band image (NIR) are aligned at the sub-pixel level. The specific process is as follows: 1. Feature point matching: Extract SIFT feature points from RGB and NIR images respectively and calculate the feature descriptor matrix and ,in , is the number of feature points, and 128 is the descriptor dimension. The matching point pairs are selected by the two-way matching strategy. ,in is the number of valid matching pairs.

[0024] 2. Parametric transformation model: using affine transformation matrix ,in is the rotation and scaling parameter, is the translation parameter. By minimizing the reprojection error objective function , and the parameters are iteratively optimized using the Levenberg-Marquardt algorithm.

[0025] 3. Deformable convolution compensation: Designing deformable convolution kernels for non-rigid deformation residues ,in is the weight coefficient, For dynamic offset, the offset field is predicted by a lightweight neural network , and finally generate a pixel-aligned two-channel image set .

[0026] Step 2: Separable convolutional network extracts brightness and color components: Decouple brightness and color features through a two-branch separable convolutional network: 1. Luminance component extraction: Design a single-channel convolution branch , its convolution kernel Using Gaussian initialization, through the formula Extract brightness features, where is the image after RGB channel grayscale, is the ReLU activation function.

[0027] 2. Color component extraction: designing a three-channel convolution branch , its convolution kernel Using Xavier initialization, through the formula Extract color features, where BN is a batch normalization operation.

[0028] 3. Feature Decoupling Constraint: Introducing Orthogonality Loss , forcing the independence of brightness and color features, and finally generating a decoupled intermediate feature map .

[0029] Step 3: Generate a vegetation activity mask with continuous edges: Combine near-infrared spectral reflectance with morphological processing to generate a precise mask: 1. Initial mask generation: Calculate the NIR channel reflectance difference map ,in is the mean and standard deviation within the local window. Generate an initial mask, is the dynamic threshold coefficient.

[0030] 2. Morphological Closure: Designing Asymmetric Structural Elements , perform the closing operation ,in For the expansion operation, For etching operations, small holes are eliminated and edges are smoothed.

[0031] Step 4: Channel weighted fusion outputs a standardized image set: Multimodal features are fused through a mask-guided weighted strategy: 1. Feature map normalization: Perform instance normalization ,in are the mean and variance of the channel dimension, .

[0032] 2. Mask weighted fusion: defining fusion weights ,in is the magnification factor, is the offset, which is multiplied pixel by pixel Generate enhanced feature maps.

[0033] 3. Standardized output: Through the deconvolution layer Mapping to a standardized image set , and introduce perceptual loss Optimize network parameters, where A feature extractor for the pre-trained VGG network, These are manually annotated standard images.

[0034] This embodiment eliminates geometric deviations between modalities through spatial registration, uses separable convolution to decouple spectral-spatial features, and generates accurate masks in combination with spectral reflectance. Finally, it achieves standardization enhancement through dynamic weighted fusion. This solves the problem of insufficient standardization accuracy caused by modal misalignment, feature aliasing, and rough masks in traditional methods, improves the separation between vegetation areas and non-vegetation areas, and lays a high signal-to-noise ratio data foundation for subsequent feature extraction.

[0035] As one example, the method extracts spectral reflectance features and texture features from the standardized image set, combines them with the vegetation activity mask, and performs regional constraint fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the regional constraint fusion suppresses noise interference in the non-vegetation area of ​​the target green plant through mask weighting, including the following sub-steps: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

[0036] In this embodiment, the visible light image and the near-infrared band image in the standardized image set are first processed separately, and the original texture feature map of the visible light image is extracted by directional gradient distribution analysis. The original spectral feature map of the near-infrared band image is obtained by spectral reflectance distribution modeling, and image features are extracted from different dimensions; then, with the help of the optimized vegetation activity mask as the spatial weight matrix, the original texture and spectral feature maps are subjected to pixel-by-pixel mask weighting operation to suppress the noise interference in the non-vegetation area in a targeted manner and realize the spatial purification of the feature map; then, the dual-channel attention mechanism is used to calculate the channel significance weights of the purified texture and spectral feature maps respectively, and highlight the key feature channels; finally, based on the obtained weight vector, the texture and spectral feature maps are cross-modal channel fused to generate a spatiotemporal correlation feature tensor. In summary, this embodiment effectively integrates the texture and spectral information of green plant images through multi-dimensional feature extraction, precise denoising, channel weight optimization and cross-modal fusion, reduces interference from non-target areas, enhances the discriminability and spatiotemporal correlation of features, and provides a more valuable feature basis for subsequent accurate judgment of the flowering stage of green plants based on deep learning, significantly improving the accuracy and reliability of flowering period monitoring.

[0037] The specific working process of this embodiment is as follows: (1) Original feature map extraction: Perform directional gradient distribution analysis on the visible light image in the standardized image set, using a multi-directional gradient convolution kernel (in is the convolution kernel size, Indicates the number of gradient directions, for example, Represents 8 different directions; Usually an odd number, such as 3, 5, etc., is used to ensure central symmetry) for convolution operation. Suppose the visible light image is ( is the image height, is the image width, is the number of channels, for visible light Figure 1 General , namely RGB three channels), through convolution operation in represents the convolution operation, For the The convolution kernel in each direction) obtains the original texture feature map ( is the height and width of the convolved image, determined by parameters such as the convolution kernel size and stride. This operation extracts edge and texture information from the image in multiple directions. Compared with traditional single-directional gradient calculation, it can more comprehensively capture the texture features of the image.

[0038] At the same time, the near-infrared band map in the standardized image set Model the spectral reflectance distribution. Using the local polynomial fitting method, in each local area of ​​the image (set as Size of the window, It can be set according to image resolution and actual needs, such as or 7), fitting a polynomial function by the least squares method ( is the degree of the polynomial, which can be set according to the complexity of the spectral reflectance change, such as ; is the spectral wavelength variable, are polynomial coefficients). By solving (in , is the pixel set in the local window) to obtain the polynomial coefficients, and then generate the original spectral feature map , each channel corresponds to a polynomial coefficient of a different order, which can describe the distribution characteristics of the spectral reflectance in more detail.

[0039] (2) Spatial purification treatment: The optimized vegetation active mask (The value range is 0 to 1, 1 represents vegetation area, 0 represents non-vegetation area) as the spatial weight matrix, the original texture feature map And the original spectral characteristics Perform pixel-by-pixel mask weighting operation. The specific formula is: ,in Represents element-by-element multiplication. Through this operation, the features of the non-vegetation area (mask value 0) are suppressed, while the features of the vegetation area (mask value 1) are retained, thus obtaining the texture feature map after spatial purification. and spectral characteristics , effectively reducing noise interference in non-vegetation areas, allowing subsequent processing to focus more on green plant characteristics.

[0040] (3) Channel saliency weight calculation: The texture feature maps after spatial purification are calculated separately through the dual-channel attention mechanism. and spectral characteristics The channel saliency weight of the texture feature map , first perform a global average pooling operation on it, and get , , and then perform feature transformation through a two-layer fully connected network: ; in is the activation function (such as Sigmoid function), is the weight matrix ( is the dimension after dimensionality reduction, which can be set according to actual conditions, such as ), is the bias vector. Finally, we get the texture feature channel weight vector .

[0041] Similarly, for the spectral characteristic diagram , after similar global average pooling and two-layer fully connected network operations, the spectral feature channel weight vector is obtained ,in , is the weight matrix ( is the dimension after dimensionality reduction, which can be set according to actual conditions, such as ) 、 is the bias vector. This dual-channel attention mechanism enhances the focus on key feature channels by learning the importance of different channels.

[0042] (4) Cross-modal channel fusion: based on texture feature channel weight vector and spectral feature channel weight vector , the texture feature map after space purification and spectral characteristics Perform cross-modal channel fusion. The specific formula is: ,in Represents the weighted operation on the channel dimension, that is, the corresponding channels are multiplied and then added. Through this fusion operation, the texture features and spectral features are organically combined at the channel level to generate a spatiotemporal correlation feature tensor. ( According to the size change after the fusion operation), it provides richer and more representative feature information for subsequent green plant flowering period monitoring.

[0043] As one example, mapping the climate parameters currently collected by the environmental sensor installed on the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism, and generating an environmentally aware spatiotemporal decision feature vector includes the following sub-steps: Characterize the climate parameters currently collected by the environmental sensor installed on the target green plant to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters; The spatiotemporal correlation feature tensor is linearly transformed to generate a cross-modal query vector, and the multi-dimensional environment feature vector is mapped into a key vector and a value vector respectively; Calculating a similarity matrix between the cross-modal query vector and the key vector to generate an environmentally aware attention weight; The value vector is weightedly aggregated according to the attention weight, and the spatiotemporal decision feature vector is output.

[0044] In this embodiment, climate parameters such as temperature, humidity, and light intensity are first collected by environmental sensors installed at the target green plants, and feature encoding is performed to obtain a multi-dimensional environmental feature vector to quantify environmental information. Then, a linear transformation is performed on the previously generated spatiotemporal correlation feature tensor to obtain a cross-modal query vector. At the same time, the multi-dimensional environmental feature vector is mapped into a key vector and a value vector to prepare for subsequent interaction. Then, a similarity matrix between the cross-modal query vector and the key vector is calculated to generate an environmental perception attention weight to clarify the degree of correlation between environmental factors and image features. Finally, the value vector is weightedly aggregated based on the attention weight to generate an environmental perception spatiotemporal decision feature vector. In summary, this embodiment breaks the isolation of image features and environmental factors, realizes effective interaction between the two through a cross-modal cross-attention mechanism, fully considers the impact of environmental factors on the flowering period of green plants, and enables the final generated feature vector to comprehensively reflect the growth status and environment of the green plants, thereby improving the comprehensiveness and accuracy of green plant flowering period monitoring and providing more reliable feature support for subsequent flowering stage judgment.

[0045] To facilitate understanding of this embodiment, the following detailed description is given here: First, the environmental sensor collects the temperature of the target green plant in real time ,humidity and light intensity Parameters. Because these three parameters have different dimensions and numerical ranges, they must first be standardized. A standardization method based on dynamic thresholds is used, and the dynamic thresholds are calculated using the quantiles of historical data.

[0046] Set the dynamic lower limit threshold of temperature and upper threshold They are the 5th and 95th percentiles of the historical temperature data, respectively. The threshold values ​​of humidity and light intensity are calculated in the same way, and we can get and Among them, the standardization process is as follows: ; in, It is an adaptive adjustment coefficient, which is dynamically adjusted according to seasonal changes. For example, in summer, the weight of temperature on flowering period increases, so it can be increased appropriately. The value of is determined by correlation analysis between historical flowering data and climate parameters.

[0047] Then, in order to extract the high-order relationship between climate parameters, the multidimensional environmental feature vector is constructed using the tensor product method. : ; in, Represents the tensor product operation, which expands the one-dimensional standardized parameters into a nine-dimensional feature vector and fully explores the interaction information between parameters.

[0048] Then perform vector mapping: Let the spatiotemporal correlation feature tensor obtained above be ,in is the tensor height, is the width, is the number of channels.

[0049] Generating cross-modal query vectors: Correlating spatiotemporal feature tensors Perform pyramid pooling operation and divide it into three sub-regions of different scales, namely 、 Perform average pooling on each sub-region to obtain feature vectors of different scales .

[0050] After concatenating these feature vectors, dimensionality reduction is performed through a two-layer fully connected network: ; in, is the Swish activation function, is the weight matrix, is the bias vector, is the middle-level dimension, To query the vector dimensions, these parameters are updated via backpropagation during model training.

[0051] Generate key vector and value vector: multidimensional environment feature vector Generate key vectors through different linear transformations Sum vector : , ,in, is the weight matrix, is the bias vector, and are the dimensions of the key vector and value vector, respectively, which are determined by hyperparameter optimization.

[0052] Next, calculate the similarity matrix and generate attention weights: use the improved cosine similarity combined with the local sensitive hashing (LSH) method to calculate the cross-modal query vector With key vector similarity.

[0053] Will and Locality sensitive hash function and Map into hash buckets to reduce computational complexity: ,in, is a random projection vector, is the offset, is the hash bucket width. These parameters are determined through multiple experiments and optimization.

[0054] For matching vector pairs in the hash bucket, calculate the improved cosine similarity: ,in, To prevent the denominator from being zero, The weight coefficient is dynamically adjusted according to the type and growth stage of the green plant, and is predicted by the classification model trained with historical data.

[0055] The similarity matrix After processing by the Softmax function, the attention weight matrix of environmental perception is generated : .

[0056] Finally, weighted aggregation generates the spatiotemporal decision feature vector: according to the attention weight matrix Pair value vector Perform weighted aggregation: ,in, This is the final generated spatiotemporal decision feature vector of environmental perception, which integrates the information of spatiotemporal correlation characteristics and environmental characteristics, and provides a key basis for the subsequent judgment of the flowering stage of green plants.

[0057] As one example, the spatiotemporal decision feature vector is input into a pre-built cascade residual classifier, and the probability distribution of the flowering stage of the target green plant is generated through the cross-layer residual connection of the cascade residual classifier, including the following sub-steps: Inputting the spatiotemporal decision feature vector into the first residual module of a pre-built cascade residual classifier, extracting local features through a convolutional layer, and performing a residual connection between the extracted results and the original input features to generate first-level enhanced features; Inputting the first-level enhanced features into the second residual module in the cascaded residual classifier, generating weight coefficients for each feature channel through a channel attention mechanism, and performing dynamic pooling processing on the feature map to generate second-level enhanced features; The second-level enhanced features are subjected to full connection layer mapping and probability normalization processing to output the flowering stage probability distribution.

[0058] In this embodiment, first, the spatiotemporal decision feature vector is input into the first residual module of the cascade residual classifier, and the convolution layer is used to extract local subtle features. The extracted results are combined with the original input features through residual connections. While avoiding the vanishing gradient, the expressive power of the features is enhanced to generate the first-level enhanced features. Then, the first-level enhanced features are sent to the second residual module, and the weight coefficients of each feature channel are calculated with the help of the channel attention mechanism to highlight the key channel information. Then, the effective features are further screened through dynamic pooling processing to generate the second-level enhanced features. Finally, the second-level enhanced features are mapped by the full connection layer and converted into dimensions corresponding to the number of flowering stages. After probability normalization processing, the probability distribution of the target green plants in each flowering stage is output. In summary, this embodiment effectively enhances the feature extraction and fusion capabilities through the coordinated operation of the cascade residual structure and multiple mechanisms, overcomes the difficulties in deep network training, and significantly improves the accuracy and stability of the classification of the flowering stages of green plants.

[0059] The specific working process of this embodiment is as follows: Step 1: First residual module processing - local feature extraction and residual connection: Assume that the spatiotemporal decision feature vector output from the above embodiment is ,in is the feature dimension (determined by the output dimension of the cross-modal attention mechanism). The first residual module contains a deformable convolution layer and an adaptive residual connection, which is implemented as follows: 1. Deformable convolution layer feature extraction: using deformable convolution kernel ,in is the number of output channels (dynamically set according to the number of flowering stages, e.g. , is the total number of flowering stages), is the convolution kernel space size (such as represents a 3×3 deformable convolution kernel). Deformable convolution is achieved by learning the offset The traditional grid sampling points are offset using the formula: ,in, is the current pixel coordinate, is the coordinate of the traditional convolution kernel sampling point, is the learnable offset, This operation enhances the ability to capture irregular plant texture features by dynamically adjusting the sampling position.

[0060] 2. Non-linear activation and residual connection: convolution output Apply the improved Swish-B activation function: ,in It is a channel-level learnable parameter that optimizes the nonlinear feature expression by adaptively adjusting the shape of the activation function curve.

[0061] Due to input Convolution output Dimensions may not be consistent (e.g. ), using residual connections with weights: ,in is the residual connection weight matrix, which is learned synchronously with the main network parameters through back propagation to ensure effective fusion of cross-layer information.

[0062] Step 2: Second residual module processing - channel attention and dynamic pooling: the first level enhanced features Enter the second residual module, which further enhances feature discriminability through a dual-channel attention mechanism and adaptive dynamic pooling: 1. Channel attention weight generation: First, Perform global mean-variance pooling to generate channel-level statistics: ,in is the channel index, is the feature map space size (if the input is a vector ).

[0063] Then the channel weights are generated through the multi-layer perceptron (MLP) + gating mechanism : ,in 、 is the weight matrix, To reduce the dimension, is the bias vector, which automatically suppresses irrelevant channel noise through learning.

[0064] 2. Channel weighting and dynamic pooling: Perform channel weighting: ,in It represents the element-by-element multiplication of the channel dimension. Then adaptive threshold dynamic pooling is used to dynamically adjust the pooling area according to the feature distribution: ,in is the adaptive coefficient (determined by training with historical flowering period data), The feature value is greater than the threshold A collection of pixels, is the number of elements in the set. This operation focuses on high-discriminative areas to avoid information loss in traditional pooling.

[0065] Step 3: Fully connected layer mapping and probability normalization: The second level enhanced features Input classification head, including cascaded fully connected layers and temperature-scaled softmax, the specific steps are as follows: 1. Multi-layer fully connected mapping: First, the dimension is transformed through the first fully connected layer: ,in is the weight matrix ( is the middle dimension), is the bias vector.

[0066] Then pass the second fully connected layer to generate the unnormalized score: ,in is the weight matrix, is the bias vector, It is the total number of flowering stages (such as bud stage, bud stage, flowering stage, withering stage, etc.).

[0067] 2. Temperature Scaling Probability Normalization: Introducing Temperature Parameters Adjust the sharpness of the Softmax distribution: , ,in Dynamic optimization is performed by minimizing the cross-entropy loss of the validation set to ensure that the probability distribution conforms to the uncertainty characteristics of the actual flowering stage.

[0068] In summary, this embodiment solves the problem of insufficient feature extraction of irregular plant morphology by traditional convolution by learning sampling offsets and cross-layer weights, and residual connections effectively alleviate the vanishing gradient of deep networks. By combining mean-variance pooling and gating mechanisms to generate channel weights, dynamic pooling adaptively selects effective areas based on feature distribution, enhancing the focus on environmentally sensitive features. In addition, the probability distribution is optimized by adjustable temperature parameters to adapt to the differences in inter-class similarity of different plant varieties during the flowering stages, thereby improving classification robustness.

[0069] As one example, the method of fusing the flowering stage probability distribution and historical flowering data based on a unidirectional temporal gating network to generate the flowering period monitoring result of the target green plant includes the following sub-steps: Splicing the current flowering stage probability distribution and historical flowering stage data in chronological order into a time series input sequence; The temporal input sequence is modeled through a unidirectional gated recurrent unit, the forgetting ratio of the historical state is controlled by a reset gate, and the fusion weight of the current input is adjusted through an update gate to generate a temporal fusion feature; Perform classification decisions on the time series fusion features and output the flowering period monitoring results.

[0070] In this embodiment, first, the current probability distribution of the flowering stage is spliced ​​with the historical flowering data in chronological order to construct a time series input sequence, and the flowering information of different time nodes is integrated to provide a data basis for analyzing the changing trend of the flowering period; then, the time series input sequence is modeled through a one-way gated recurrent unit, and the reset gate is used to flexibly control the forgetting ratio of the historical state to avoid interference from invalid historical information. At the same time, the fusion weight of the current input is adjusted with the help of the update gate, and the new information and historical information are adaptively fused to generate a time series fusion feature containing time dimension features; finally, the time series fusion feature is classified and decided, and the final flowering period monitoring result is output. In summary, this embodiment fully utilizes the laws and trends contained in the historical flowering period data through deep mining and intelligent processing of time series data, and combines the current probability distribution of flowering period to effectively improve the accuracy and foresight of flowering period monitoring, and can more timely and accurately predict changes in the flowering stage of green plants.

[0071] To facilitate understanding of this embodiment, the following detailed description is given here: Step 1: Time series input sequence construction: Assume that the probability distribution of the current target green plant flowering stage is ,in The total number of flowering stages (such as bud stage, bud stage, flowering stage, withering stage, etc.). Historical flowering stage data includes past probability distribution sequence of time steps , each .

[0072] To integrate current data with historical data, we first perform time alignment and feature enhancement on the historical data. Since data collected at different times may have different sampling frequencies, we use an improved version of the dynamic time warping (DTW) algorithm for alignment. The formula is as follows: ,in Indicates the historical data and The alignment distance of the probability distribution of time steps, is the Euclidean distance. Find the optimal alignment sequence by backtracking the path .

[0073] Then, trend feature extraction is performed on the aligned historical data. The probability distribution change rate of adjacent time steps is calculated: ,in is the time interval between adjacent time steps. The rate of change feature is spliced ​​with the original probability distribution to obtain enhanced historical data .

[0074] Finally, the current probability distribution With enhanced historical data Splice in time order to generate a time series input sequence : .

[0075] Step 2: Unidirectional Gated Recurrent Unit (UGRU) Modeling: Input the time series into the sequence The input is fed into a custom unidirectional gated recurrent unit (UGRU). UGRU uses a dual reset gate and dynamic update gate mechanism to achieve selective forgetting of historical states and adaptive fusion of current inputs. The specific formula is as follows: 1. Double reset gate calculation: the first reset gate Used to control the forgetting of long-term historical states: ; Second reset gate Used to control forgetting of short-term history states: ; in is the Sigmoid activation function, is the input weight matrix, is the hidden state weight matrix, is the bias vector, is the hidden layer dimension, determined by hyperparameter optimization.

[0076] 2. Dynamic update gate calculation: Considering the importance of data at different flowering stages, an update gate enhanced by the attention mechanism is introduced. : ; in is the weight matrix, is the bias vector, is the attention weight vector, Represents element-wise multiplication.

[0077] 3. Hidden state update: Combine the double reset gate and the dynamic update gate to calculate the current hidden state : ; in is the weight matrix, is the bias vector. After time steps, the final hidden state This is the time series fusion feature.

[0078] Step 3: Classification decision and result output: Fusion of time series features The data is input into the multi-layer perceptron (MLP) classifier to predict the flowering period monitoring results. The MLP consists of two fully connected layers: 1. The first fully connected layer: ; in is the weight matrix, is the bias vector, is the intermediate layer dimension, determined by hyperparameter optimization, and $\text{ReLU}$ is the rectified linear unit activation function.

[0079] 2. Second fully connected layer: ; in is the weight matrix, is the bias vector.

[0080] Output of the second layer Perform fuzzy probability normalization processing to obtain the final probability distribution of flowering period monitoring results : ; in This is a correction factor calculated based on historical data fluctuations. It is used to adjust the confidence level of the probability distribution to ensure that the output is more consistent with the actual flowering period variation. The flowering stage corresponding to the highest probability is the final flowering period monitoring result.

[0081] In summary, this embodiment addresses the temporal inconsistency issue of historical data through an improved DTW algorithm and enhances the dynamic nature of time series information through rate-of-change features. Furthermore, the UGRU implements layered forgetting of historical states through dual reset gates. Combined with an attention mechanism, it dynamically adjusts update gate weights to effectively integrate flowering period data from multiple time scales. Furthermore, a probabilistic calculation method using correction coefficients enhances the adaptability of the results to data fluctuations and improves the reliability of monitoring results.

[0082] See also Figure 2, is a schematic diagram of a green plant flowering period monitoring device based on deep learning provided by one embodiment of the present invention. The green plant flowering period monitoring device based on deep learning includes: An image generation module 10 is configured to acquire a current time-series image set of a target plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the image set, and generate a standardized image set comprising a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target plant using a learnable convolution kernel, and generates a vegetation activity mask of the target plant based on the spectral reflectance of the near-infrared channel; A feature extraction module 11 is configured to extract spectral reflectance features and texture features from the standardized image set, combine them with the vegetation activity mask, and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; a vector generation module 12 for mapping the climate parameters currently collected by the environmental sensor of the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception; The probability prediction module 13 is used to input the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generate the flowering stage probability distribution of the target green plant through the cross-layer residual connection of the cascade residual classifier; The flowering period analysis module 14 is used to generate a flowering period monitoring result of the target green plant by fusing the flowering period probability distribution and the historical flowering period data based on a unidirectional temporal gating network.

[0083] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The embodiment of the present invention obtains a time-series image set of the target green plant through an image acquisition device, performs cross-modal dynamic enhancement processing, uses a learnable convolution kernel to separate the brightness and color components, generates a vegetation activity mask based on the spectral reflectance of the near-infrared channel, and obtains a standardized image set containing visible light and near-infrared band maps, thereby quantifying the physiological activity state of the vegetation; then extracts the spectral reflectance characteristics and texture characteristics of the standardized image set, combines them with the vegetation activity mask, uses the channel attention mechanism to perform regional constraint fusion, suppresses noise interference in non-vegetation areas, and generates a spatiotemporal correlation feature tensor; maps the climate parameters collected by the environmental sensor into an environmental feature vector, interacts with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism, generates an environmentally aware spatiotemporal decision feature vector, and dynamically monitors the flowering period in combination with the environmental parameters; finally, inputs the spatiotemporal decision feature vector into a cascade residual classifier, uses cross-layer residual connections to generate a probability distribution of the flowering stage, and then fuses the historical flowering data through a unidirectional temporal gating network to obtain a flowering period monitoring result. In summary, the embodiments of the present invention significantly improve the accuracy and environmental robustness of green plant flowering period monitoring in complex environments through the collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion and environmental perception decision-making.

[0084] As one example, the image generation module is specifically configured to: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

[0085] As one example, the feature extraction module is specifically used to: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

[0086] It can be understood that the relevant embodiments of the above-mentioned green plant flowering period monitoring device based on deep learning can refer to the contents of the above-mentioned green plant flowering period monitoring method embodiment based on deep learning, and will not be described in detail here.

[0087] See also Figure 3 , is a schematic diagram of a green plant flowering period monitoring system based on deep learning provided by an embodiment of the present invention. The green plant flowering period monitoring system based on deep learning of this embodiment includes: a processor 100, a memory 101, and a computer program stored in the memory 101 and executable on the processor 100, such as a green plant flowering period monitoring program based on deep learning. When the processor 100 executes the computer program, the steps in each of the above-mentioned embodiments of the green plant flowering period monitoring method based on deep learning are implemented. Alternatively, when the processor 100 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0088] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the deep learning-based green plant flowering period monitoring system.

[0089] The green plant flowering period monitoring system based on deep learning can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The green plant flowering period monitoring system based on deep learning may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagram is only an example of a green plant flowering period monitoring system based on deep learning, and does not constitute a limitation of the green plant flowering period monitoring system based on deep learning. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the green plant flowering period monitoring system based on deep learning may also include input and output devices, network access devices, buses, etc.

[0090] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the deep learning-based green plant flowering period monitoring system, and utilizes various interfaces and lines to connect the various parts of the deep learning-based green plant flowering period monitoring system.

[0091] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the deep learning-based green plant flowering period monitoring system by running or executing the computer programs and / or modules stored in the memory and accessing the data stored in the memory. The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0092] If the integrated modules / units of the deep learning-based green plant flowering period monitoring system are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, computer memory, read-only memory (ROM), random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0093] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0094] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A green plant flowering period monitoring method based on deep learning, characterized in that: include: Acquire a current time-series image set of a target green plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the image set, and generate a standardized image set including a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolution kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel; Extracting spectral reflectance features and texture features from the standardized image set, combining them with the vegetation activity mask and performing region-constrained fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; Mapping the climate parameters currently collected by the environmental sensor installed on the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate an environmental perception spatiotemporal decision feature vector; Inputting the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generating a probability distribution of the flowering stage of the target green plant through the cross-layer residual connection of the cascade residual classifier; The flowering stage probability distribution and historical flowering stage data are fused based on a unidirectional temporal gating network to generate the flowering stage monitoring result of the target green plant.

2. The green plant flowering period monitoring method based on deep learning according to claim 1, characterized in that: The method of acquiring a current time-series image set of a target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the image set, and generating a standardized image set including a visible light image and a near-infrared band image includes the following sub-steps: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

3. The green plant flowering period monitoring method based on deep learning according to claim 2, characterized in that: The method extracts spectral reflectance features and texture features from the standardized image set, combines them with the vegetation activity mask, and performs regional constraint fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the regional constraint fusion suppresses noise interference in the non-vegetation area of ​​the target green plant through mask weighting, and includes the following sub-steps: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

4. The green plant flowering period monitoring method based on deep learning according to claim 3, characterized in that: The process of mapping the climate parameters currently collected by the environmental sensor of the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism, and generating an environmentally-aware spatiotemporal decision feature vector includes the following sub-steps: Characterize the climate parameters currently collected by the environmental sensor installed on the target green plant to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters; The spatiotemporal correlation feature tensor is linearly transformed to generate a cross-modal query vector, and the multi-dimensional environment feature vector is mapped into a key vector and a value vector respectively; Calculating a similarity matrix between the cross-modal query vector and the key vector to generate an environmentally aware attention weight; The value vector is weightedly aggregated according to the attention weight, and the spatiotemporal decision feature vector is output.

5. The green plant flowering period monitoring method based on deep learning according to claim 4, characterized in that: The step of inputting the spatiotemporal decision feature vector into a pre-built cascade residual classifier and generating a probability distribution of the flowering stage of the target green plant through the cross-layer residual connection of the cascade residual classifier comprises the following sub-steps: Inputting the spatiotemporal decision feature vector into the first residual module of a pre-built cascade residual classifier, extracting local features through a convolutional layer, and performing a residual connection between the extracted results and the original input features to generate first-level enhanced features; Inputting the first-level enhanced features into the second residual module in the cascaded residual classifier, generating weight coefficients for each feature channel through a channel attention mechanism, and performing dynamic pooling processing on the feature map to generate second-level enhanced features; The second-level enhanced features are subjected to full connection layer mapping and probability normalization processing to output the flowering stage probability distribution.

6. The green plant flowering period monitoring method based on deep learning according to claim 5, characterized in that: The method of fusing the flowering stage probability distribution and historical flowering data based on a unidirectional temporal gating network to generate the flowering period monitoring result of the target green plant includes the following sub-steps: Splicing the current flowering stage probability distribution and historical flowering stage data in chronological order into a time series input sequence; The temporal input sequence is modeled through a unidirectional gated recurrent unit, the forgetting ratio of the historical state is controlled by a reset gate, and the fusion weight of the current input is adjusted through an update gate to generate a temporal fusion feature; Perform classification decisions on the time series fusion features and output the flowering period monitoring results.

7. A green plant flowering period monitoring device based on deep learning, characterized in that: include: An image generation module is configured to acquire a current time-series image set of a target plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the image set, and generate a standardized image set comprising a visible light image and a near-infrared band image; wherein the dynamic enhancement processing separates the brightness and color components of the target plant using a learnable convolution kernel and generates a vegetation activity mask for the target plant based on the spectral reflectance of the near-infrared channel; A feature extraction module is configured to extract spectral reflectance features and texture features from the standardized image set, combine them with the vegetation activity mask, and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein the region-constrained fusion suppresses noise interference in non-vegetation areas of the target green plants through mask weighting; a vector generation module for mapping the climate parameters currently collected by the environmental sensor installed on the target green plant into an environmental feature vector, interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception; A probability prediction module, configured to input the spatiotemporal decision feature vector into a pre-built cascade residual classifier, and generate a probability distribution of the flowering stage of the target green plant through a cross-layer residual connection of the cascade residual classifier; The flowering period analysis module is used to generate the flowering period monitoring results of the target green plant by fusing the flowering stage probability distribution and historical flowering period data based on a unidirectional temporal gating network.

8. The green plant flowering period monitoring device based on deep learning according to claim 7, characterized in that: The image generation module is specifically used for: Performing spatial registration processing on the visible light band image and the near-infrared band image of the time-series image set to generate a pixel-aligned dual-channel image set; Inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through the first convolution branch, extracting color component features through the second convolution branch, and generating an intermediate feature map in which brightness and color are decoupled; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band image, and performing a morphological closing operation on the initial mask to generate a vegetation activity mask with continuous edges; The intermediate feature map obtained by decoupling brightness and color is fused with the vegetation activity mask in a channel-weighted manner to output the standardized image set.

9. The green plant flowering period monitoring device based on deep learning according to claim 8, characterized in that: The feature extraction module is specifically used for: Performing directional gradient distribution analysis on the visible light image in the standardized image set to extract the original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band image in the standardized image set to generate the original spectral feature map; Using the optimized vegetation activity mask as a spatial weight matrix, performing pixel-by-pixel mask weighting operations on the original texture feature map and the original spectral feature map, respectively, suppressing characteristic noise in non-vegetation areas, and obtaining a spatially purified texture feature map and spectral feature map; The channel saliency weights of the texture feature map and the spectral feature map after the spatial purification are calculated respectively through a dual-channel attention mechanism to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map to generate the associated feature tensor.

10. A green plant flowering period monitoring system based on deep learning, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the green plant flowering period monitoring method based on deep learning as described in any one of claims 1 to 6.

Images