Green plant flowering period monitoring method, device and system based on deep learning

Abstract

The application discloses a green plant flowering period monitoring method, device and system based on deep learning, cross-modal dynamic enhancement of visible light and near-infrared images is realized through a learnable convolution kernel, a vegetation activity mask is generated to accurately locate a vegetation area, channel attention mechanism is combined with the mask to weight and fuse spectral and texture features, and non-vegetation noise interference is inhibited, cross-modal cross-attention is used to dynamically associate climate parameters and image features, nonlinear influence of environmental changes on the flowering period is captured, finally, a cascaded residual classifier and a one-way time sequence gating network are used to fuse multi-scale features and time sequence rules, and high-precision and strong-robust green plant flowering period monitoring in a complex environment is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus and system for monitoring the flowering period of green plants based on deep learning. Background Technology

[0002] In the current field of green plant flowering period monitoring, common monitoring methods mainly include manual observation and methods based on traditional sensors or simple image recognition technology. Manual observation requires staff to regularly go to the site to observe the morphological changes of green plants with the naked eye and record relevant information about the flowering period. Although this method can obtain data to a certain extent, the process is extremely cumbersome, requires a large investment of manpower and time, and is affected by the subjective perception and experience differences of the observers, making it difficult to guarantee the accuracy of the data.

[0003] With technological advancements, image recognition-based methods for monitoring flowering periods have emerged. These methods primarily utilize image processing algorithms (such as color space conversion and edge detection) to extract features from buds or petals, and then classify the flowering stage using classifiers like support vector machines and random forests. However, these methods still have significant drawbacks: First, they rely on a single visible light band, failing to quantify the physiological activity of vegetation through spectral information (such as near-infrared reflectance), leading to misclassification of withered and healthy areas. Second, they fail to effectively suppress noise interference from non-vegetated areas (such as soil and flowerpots), introducing a large amount of irrelevant background information during feature extraction, affecting the analysis of green plant flowering period monitoring. Third, they do not incorporate environmental parameters to dynamically monitor changes in flowering periods, making it difficult to address the delayed effects of sudden environmental changes on flowering.

[0004] In summary, existing technologies are not accurate enough for monitoring the flowering period of green plants in complex scenarios. Summary of the Invention

[0005] This invention provides a method, device, and system for monitoring the flowering period of green plants based on deep learning, which can improve the accuracy of flowering period monitoring in complex scenarios.

[0006] One embodiment of the present invention provides a deep learning-based method for monitoring the flowering period of green plants, comprising:

[0007] The current time-series image set of the target green plant is acquired through an image acquisition device, and cross-modal dynamic enhancement processing is performed on the time-series image set to generate a standardized image set containing visible light image and near-infrared band image; wherein, the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolutional kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel.

[0008] Spectral reflectance features and texture features are extracted from the standardized image set. The vegetation activity mask is combined with the spectral reflectance features and texture features to perform region-constrained fusion through a channel attention mechanism to generate a spatiotemporal correlation feature tensor. The region-constrained fusion suppresses noise interference in the non-vegetation areas of the target green plant through mask weighting.

[0009] The climate parameters currently collected by the environmental sensor set on the target green plant are mapped into an environmental feature vector. The environmental feature vector is then interacted with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception.

[0010] The spatiotemporal decision feature vector is input into a pre-constructed cascaded residual classifier, and the probability distribution of the flowering stage of the target green plant is generated through the cross-layer residual connection of the cascaded residual classifier.

[0011] The flowering period monitoring results of the target green plant are generated by fusing the probability distribution of the flowering period stage with historical flowering period data using a one-way temporal gating network.

[0012] As an improvement to the above solution, the step of acquiring the current time-series image set of the target green plant through an image acquisition device, and performing cross-modal dynamic enhancement processing on the time-series image set to generate a standardized image set containing visible light images and near-infrared band images includes the following sub-steps:

[0013] Spatial registration processing is performed on the visible light band map and near-infrared band map of the time-series image set to generate a pixel-aligned dual-channel image set;

[0014] The dual-channel image set is input into a separable convolutional network. The brightness component features are extracted through the first convolutional branch, and the color component features are extracted through the second convolutional branch to generate an intermediate feature map that decouples brightness and color.

[0015] An initial vegetation activity mask is generated based on the spectral reflectance difference of the near-infrared band image, and a morphological closing operation is performed on the initial vegetation activity mask to generate a vegetation activity mask with continuous edges.

[0016] The intermediate feature map, which decouples brightness and color, is fused with the vegetation activity mask using channel weighting to output the standardized image set.

[0017] As an improvement to the above scheme, the extraction of spectral reflectance features and texture features from the standardized image set, combined with the vegetation activity mask, and the region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism, generates a spatiotemporal correlation feature tensor; wherein, the region-constrained fusion suppresses noise interference from non-vegetation areas of the target green plant through mask weighting, including the following sub-steps:

[0018] Orientation gradient distribution analysis is performed on the visible light images in the standardized image set to extract the original texture feature map. At the same time, spectral reflectance distribution modeling is performed on the near-infrared band images in the standardized image set to generate the original spectral feature map.

[0019] Using the vegetation activity mask as a spatial weight matrix, pixel-by-pixel mask weighting operations are performed on the original texture feature map and the original spectral feature map to suppress feature noise in non-vegetation areas, resulting in spatially purified texture feature map and spectral feature map.

[0020] The channel saliency weights of the spatially purified texture feature map and spectral feature map are calculated separately using a dual-channel attention mechanism to generate texture feature channel weight vectors and spectral feature channel weight vectors.

[0021] Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and spectral feature map to generate the associated feature tensor.

[0022] As an improvement to the above scheme, the step of mapping the climate parameters currently collected by the environmental sensor on the target green plant into an environmental feature vector, and interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception includes the following sub-steps:

[0023] The climate parameters currently collected by the environmental sensors installed on the target green plant are feature-encoded to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters;

[0024] The spatiotemporal correlation feature tensor is transformed linearly to generate a cross-modal query vector, and the multidimensional environmental feature vector is mapped to a key vector and a value vector respectively.

[0025] Calculate the similarity matrix between the cross-modal query vector and the key vector to generate environment-aware attention weights;

[0026] The value vector is weighted and aggregated according to the attention weights to output the spatiotemporal decision feature vector.

[0027] As an improvement to the above scheme, the step of inputting the spatiotemporal decision feature vector into a pre-constructed cascaded residual classifier, and generating the probability distribution of the flowering stage of the target green plant through the cross-layer residual connections of the cascaded residual classifier, includes the following sub-steps:

[0028] The spatiotemporal decision feature vector is input into the first residual module of the pre-constructed cascaded residual classifier. Local features are extracted through the convolutional layer, and the extraction results are residually connected with the original input features to generate the first-level enhanced features.

[0029] The first-level enhanced features are input into the second residual module in the cascaded residual classifier. The weight coefficients of each feature channel are generated through the channel attention mechanism, and dynamic pooling is performed on the feature map to generate the second-level enhanced features.

[0030] The second-level enhanced features are mapped and normalized using a fully connected layer to output the probability distribution of the flowering stage.

[0031] As an improvement to the above scheme, the step of generating the flowering period monitoring results of the target green plant by fusing the probability distribution of the flowering period stage with historical flowering period data based on a one-way temporal gating network includes the following sub-steps:

[0032] The probability distribution of the flowering stage is concatenated with historical flowering data in chronological order to form a time-series input sequence;

[0033] The time-series input sequence is modeled by a one-way gated loop unit, the forgetting ratio of historical states is controlled by a reset gate, and the fusion weight of the current input is adjusted by an update gate to generate time-series fusion features.

[0034] The time-series fusion features are used to make classification decisions, and the flowering period monitoring results are output.

[0035] Another embodiment of the present invention provides a deep learning-based green plant flowering period monitoring device, comprising:

[0036] The image generation module is used to acquire the current time-series image set of the target green plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the time-series image set, and generate a standardized image set containing visible light image and near-infrared band image; wherein, the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolutional kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel.

[0037] The feature extraction module is used to extract spectral reflectance features and texture features from the standardized image set, combine them with the vegetation activity mask, and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein, the region-constrained fusion suppresses noise interference in the non-vegetation areas of the target green plant through mask weighting.

[0038] The vector generation module is used to map the climate parameters currently collected by the environmental sensor set on the target green plant into an environmental feature vector. The environmental feature vector is then interacted with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception.

[0039] The probability prediction module is used to input the spatiotemporal decision feature vector into a pre-constructed cascaded residual classifier, and generate the probability distribution of the flowering stage of the target green plant through the cross-layer residual connection of the cascaded residual classifier.

[0040] The flowering period analysis module is used to generate the flowering period monitoring results of the target green plant by fusing the probability distribution of the flowering period stage with historical flowering period data based on a one-way temporal gating network.

[0041] As an improvement to the above solution, the image generation module is specifically used for:

[0042] Spatial registration processing is performed on the visible light band map and near-infrared band map of the time-series image set to generate a pixel-aligned dual-channel image set;

[0043] The dual-channel image set is input into a separable convolutional network. The brightness component features are extracted through the first convolutional branch, and the color component features are extracted through the second convolutional branch to generate an intermediate feature map that decouples brightness and color.

[0044] An initial vegetation activity mask is generated based on the spectral reflectance difference of the near-infrared band image, and a morphological closing operation is performed on the initial vegetation activity mask to generate a vegetation activity mask with continuous edges.

[0045] The intermediate feature map, which decouples brightness and color, is fused with the vegetation activity mask using channel weighting to output the standardized image set.

[0046] As an improvement to the above scheme, the feature extraction module is specifically used for:

[0047] Orientation gradient distribution analysis is performed on the visible light images in the standardized image set to extract the original texture feature map. At the same time, spectral reflectance distribution modeling is performed on the near-infrared band images in the standardized image set to generate the original spectral feature map.

[0048] Using the vegetation activity mask as a spatial weight matrix, pixel-by-pixel mask weighting operations are performed on the original texture feature map and the original spectral feature map to suppress feature noise in non-vegetation areas, resulting in spatially purified texture feature map and spectral feature map.

[0049] The channel saliency weights of the spatially purified texture feature map and spectral feature map are calculated separately using a dual-channel attention mechanism to generate texture feature channel weight vectors and spectral feature channel weight vectors.

[0050] Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and spectral feature map to generate the associated feature tensor.

[0051] Another embodiment of the present invention provides a deep learning-based green plant flowering period monitoring system, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the deep learning-based green plant flowering period monitoring method described in the above-described embodiment of the invention.

[0052] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0053] This invention acquires a time-series image set of target green plants through an image acquisition device. After cross-modal dynamic enhancement processing, a learnable convolutional kernel is used to separate the brightness and color components. A vegetation activity mask is generated based on the spectral reflectance of the near-infrared channel, resulting in a standardized image set containing visible and near-infrared band images, thereby quantifying the physiological activity state of the vegetation. Then, the spectral reflectance and texture features of the standardized image set are extracted and combined with the vegetation activity mask. A channel attention mechanism is used for region-constrained fusion to suppress noise interference in non-vegetation areas, generating a spatiotemporal correlation feature tensor. Climate parameters collected by environmental sensors are mapped into environmental feature vectors. Through cross-modal cross-attention mechanism and interaction with the spatiotemporal correlation feature tensor, a spatiotemporal decision feature vector for environmental perception is generated, thereby dynamically monitoring the flowering period in conjunction with environmental parameters. Finally, the spatiotemporal decision feature vector is input into a cascaded residual classifier. Cross-layer residual connections are used to generate a probability distribution of the flowering period stage. Then, historical flowering period data is fused through a unidirectional temporal gating network to obtain the flowering period monitoring results. In summary, the embodiments of the present invention significantly improve the accuracy and robustness of monitoring the flowering period of green plants in complex environments through a collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion, and environmental perception decision-making. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a deep learning-based method for monitoring the flowering period of green plants, provided in an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of a deep learning-based green plant flowering period monitoring device provided in an embodiment of the present invention;

[0056] Figure 3This is a schematic diagram of a deep learning-based green plant flowering period monitoring system provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] See Figure 1 This is a flowchart illustrating a deep learning-based method for monitoring the flowering period of green plants, provided in an embodiment of the present invention. The deep learning-based method for monitoring the flowering period of green plants includes the following steps:

[0059] S10, the current time-series image set of the target green plant is acquired through the image acquisition device, and cross-modal dynamic enhancement processing is performed on the time-series image set to generate a standardized image set containing visible light image and near-infrared band image; wherein, the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolutional kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel.

[0060] S11, extract the spectral reflectance features and texture features from the standardized image set, combine them with the vegetation activity mask, and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein, the region-constrained fusion suppresses noise interference in the non-vegetation areas of the target green plant through mask weighting.

[0061] S12, the climate parameters currently collected by the environmental sensor set on the target green plant are mapped into an environmental feature vector, and the environmental feature vector is interacted with the spatiotemporal correlation feature tensor through a cross-modal cross attention mechanism to generate a spatiotemporal decision feature vector for environmental perception.

[0062] S13, input the spatiotemporal decision feature vector into the pre-constructed cascaded residual classifier, and generate the flowering stage probability distribution of the target green plant through the cross-layer residual connection of the cascaded residual classifier;

[0063] S14. Based on a one-way temporal gating network, the probability distribution of the flowering stage and historical flowering data are fused to generate the flowering period monitoring results of the target green plant.

[0064] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0065] This invention acquires a time-series image set of target green plants through an image acquisition device. After cross-modal dynamic enhancement processing, a learnable convolutional kernel is used to separate the brightness and color components. A vegetation activity mask is generated based on the spectral reflectance of the near-infrared channel, resulting in a standardized image set containing visible and near-infrared band images, thereby quantifying the physiological activity state of the vegetation. Then, the spectral reflectance and texture features of the standardized image set are extracted and combined with the vegetation activity mask. A channel attention mechanism is used for region-constrained fusion to suppress noise interference in non-vegetation areas, generating a spatiotemporal correlation feature tensor. Climate parameters collected by environmental sensors are mapped into environmental feature vectors. Through cross-modal cross-attention mechanism and interaction with the spatiotemporal correlation feature tensor, a spatiotemporal decision feature vector for environmental perception is generated, thereby dynamically monitoring the flowering period in conjunction with environmental parameters. Finally, the spatiotemporal decision feature vector is input into a cascaded residual classifier. Cross-layer residual connections are used to generate a probability distribution of the flowering period stage. Then, historical flowering period data is fused through a unidirectional temporal gating network to obtain the flowering period monitoring results. In summary, the embodiments of the present invention significantly improve the accuracy and environmental robustness of monitoring the flowering period of green plants in complex environments through a collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion, and environmental perception decision-making.

[0066] As one example, the step of acquiring a current time-series image set of the target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the time-series image set, and generating a standardized image set containing visible light and near-infrared band images includes the following sub-steps:

[0067] Spatial registration processing is performed on the visible light band map and near-infrared band map of the time-series image set to generate a pixel-aligned dual-channel image set;

[0068] The dual-channel image set is input into a separable convolutional network. The brightness component features are extracted through the first convolutional branch, and the color component features are extracted through the second convolutional branch to generate an intermediate feature map that decouples brightness and color.

[0069] An initial vegetation activity mask is generated based on the spectral reflectance difference of the near-infrared band image, and a morphological closing operation is performed on the initial vegetation activity mask to generate a vegetation activity mask with continuous edges.

[0070] The intermediate feature map, which decouples brightness and color, is fused with the vegetation activity mask using channel weighting to output the standardized image set.

[0071] In this embodiment, focusing on image acquisition and processing, and aiming to improve the image quality and feature accuracy of green plants, a series of innovative operations are used to generate a standardized image set. First, an image acquisition device is used to obtain the current temporal image set of the target green plant. Considering the spatial misalignment between the visible light band image and the near-infrared band image, spatial registration is used to align pixels and generate a dual-channel image set, laying the foundation for subsequent processing. Next, a separable convolutional network is used, inputting the dual-channel image set. The first convolutional branch extracts the brightness component features, and the second convolutional branch extracts the color component features, effectively decoupling brightness and color to obtain an intermediate feature map. Then, an initial vegetation activity mask is generated based on the difference in spectral reflectance of the near-infrared band image. Morphological closing operations are used to eliminate holes and discontinuous edges in the mask, generating a vegetation activity mask with continuous edges, accurately segmenting the vegetation region. Finally, the intermediate feature map with decoupled brightness and color is fused with the vegetation activity mask using channel weighting to highlight the image features of the vegetation region, suppress background noise interference, and output a standardized image set containing both visible light and near-infrared band images. In summary, this embodiment effectively improves the clarity and accuracy of plant features in images, providing high-quality data support for subsequent image-based monitoring of plant flowering periods, and enhancing the reliability and stability of monitoring results.

[0072] The specific working process of this embodiment is as follows:

[0073] Step 1: Spatial registration to generate a dual-channel image set: A spatial registration algorithm based on deformable convolution is designed to perform sub-pixel-level alignment between the visible light band image (RGB) and the near-infrared band image (NIR). The specific process is as follows:

[0074] 1. Feature point matching: Extract SIFT feature points from RGB and NIR images respectively, and calculate the feature descriptor matrix. and ,in , 128 represents the number of feature points, and 128 represents the descriptor dimension. Matching point pairs are selected using a bidirectional matching strategy. ,in For valid logarithmic matching.

[0075] 2. Parametric Transformation Model: Employing an affine transformation matrix ,in For rotation and scaling parameters, Let be the translation parameters. The objective function is to minimize the reprojection error. The parameters were iteratively optimized using the Levenberg-Marquardt algorithm.

[0076] 3. Deformable Convolution Compensation: To address non-rigid deformation residues, deformable convolution kernels are designed. ,in These are the weighting coefficients. The offset is a dynamic offset, and the offset field is predicted using a lightweight neural network. Finally, a pixel-aligned dual-channel image set is generated. .

[0077] Step 2, Separable Convolutional Network Extracts Luminance and Color Components: A two-branch separable convolutional network decouples luminance and color features.

[0078] 1. Luminance component extraction: Design a single-channel convolution branch Its convolution kernel Gaussian initialization is used, through the formula Extract brightness features, where This is the image after grayscale conversion of the RGB channels. This is the ReLU activation function.

[0079] 2. Color component extraction: Design a three-channel convolution branch. Its convolution kernel Initialize using Xavier, via formula Extract color features, where BN is a batch normalization operation.

[0080] 3. Feature decoupling constraint: Introduce orthogonality loss. By forcibly constraining the independence of brightness and color features, a decoupled intermediate feature map is ultimately generated. .

[0081] Step 3, generating a vegetation-active mask with continuous edges: A precise mask is generated by combining near-infrared spectral reflectance with morphological processing.

[0082] 1. Initial vegetation active mask generation: Calculation of NIR channel reflectance difference map ,in These represent the mean and standard deviation within a local window. An adaptive threshold formula is used. Generate an initial vegetation activity mask. This is the dynamic threshold coefficient.

[0083] 2. Morphological Closure Operation: Designing Asymmetric Structural Elements Perform closing operation ,in For expansion operation, For corrosion operations, small holes are eliminated and edges are smoothed.

[0084] Step 4, Channel-weighted fusion outputs a standardized image set: Multimodal features are fused using a mask-guided weighting strategy.

[0085] 1. Feature map normalization: For Execution instance normalization ,in Here are the mean and variance for the channel dimension. .

[0086] 2. Mask-weighted fusion: Defining the fusion weights ,in This is the magnification factor. The offset is calculated using pixel-by-pixel multiplication. Generate enhanced feature maps.

[0087] 3. Standardized output: This is achieved through a deconvolution layer. Mapped to a normalized image set And introduce perceived loss Optimize network parameters, among which For feature extractors of pre-trained VGG networks, Standard images with manual annotations.

[0088] This embodiment eliminates geometric deviations between modes through spatial registration, decouples spectral-spatial features using separable convolution, generates a precise mask by combining spectral reflectance, and finally achieves standardization enhancement through dynamic weighted fusion. This solves the problem of insufficient standardization accuracy caused by mode misalignment, feature aliasing, and mask coarseness in traditional methods, thereby improving the separation between vegetated and non-vegetated areas and laying a high signal-to-noise ratio data foundation for subsequent feature extraction.

[0089] As one example, the step of extracting spectral reflectance features and texture features from the standardized image set, combining them with the vegetation activity mask, and performing region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporally correlated feature tensor; wherein, the region-constrained fusion suppresses noise interference from non-vegetation areas of the target green plant through mask weighting, and includes the following sub-steps:

[0090] Orientation gradient distribution analysis is performed on the visible light images in the standardized image set to extract the original texture feature map. At the same time, spectral reflectance distribution modeling is performed on the near-infrared band images in the standardized image set to generate the original spectral feature map.

[0091] Using the vegetation activity mask as a spatial weight matrix, pixel-by-pixel mask weighting operations are performed on the original texture feature map and the original spectral feature map to suppress feature noise in non-vegetation areas, resulting in spatially purified texture feature map and spectral feature map.

[0092] The channel saliency weights of the spatially purified texture feature map and spectral feature map are calculated separately using a dual-channel attention mechanism to generate texture feature channel weight vectors and spectral feature channel weight vectors.

[0093] Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and spectral feature map to generate the associated feature tensor.

[0094] In this embodiment, the visible light image and near-infrared band image in the standardized image set are processed separately. The original texture feature map of the visible light image is extracted by directional gradient distribution analysis, and the original spectral feature map of the near-infrared band image is obtained by spectral reflectance distribution modeling. Image features are extracted from different dimensions. Then, using a vegetation activity mask as a spatial weight matrix, pixel-by-pixel mask weighting is performed on the original texture and spectral feature maps to specifically suppress noise interference in non-vegetation areas and achieve spatial purification of the feature maps. Next, a dual-channel attention mechanism is used to calculate the channel saliency weights of the purified texture and spectral feature maps respectively to highlight key feature channels. Finally, based on the obtained weight vector, the texture and spectral feature maps are fused across modal channels to generate a spatiotemporally correlated feature tensor. In summary, this embodiment effectively integrates the texture and spectral information of green plant images through multi-dimensional feature extraction, precise denoising, channel weight optimization, and cross-modal fusion. This reduces interference from non-target areas, enhances the discriminative power and spatiotemporal correlation of features, and provides a more valuable feature foundation for subsequent accurate judgment of green plant flowering stages based on deep learning. This significantly improves the accuracy and reliability of flowering period monitoring.

[0095] The specific working process of this embodiment is as follows:

[0096] (1) Extraction of original feature maps:

[0097] Orientation gradient distribution analysis was performed on visible light images in the standardized image set using multi-directional gradient convolution kernels. (in The kernel size is the convolution kernel size. This represents the number of gradient directions, for example, it can be set as follows: Representing 8 different directions; Odd numbers, such as 3 or 5, are usually chosen to ensure central symmetry during convolution operations. Let the visible light image be... ( Image height, Image width, For visible light, the number of channels is... Figure 1 Ban (i.e., the three channels RGB), through convolution operations in This represents the convolution operation. For the first The original texture feature map is obtained by convolution kernels in several directions. ( The height and width of the convolved image are determined by parameters such as kernel size and stride. This operation extracts edge and texture information from the image in multiple directions, capturing the image's texture features more comprehensively compared to traditional unidirectional gradient calculation.

[0098] Meanwhile, near-infrared band images from the standardized image set Modeling of spectral reflectance distribution is performed. A local polynomial fitting method is used, applying it to each local region of the image (let's call it...). Window size It can be set according to image resolution and actual needs, such as Or, within 7), fit the polynomial function using the least squares method. ( The degree is a polynomial, which can be set according to the complexity of the spectral reflectance variation, such as... ; For spectral wavelength variables, (These are the polynomial coefficients). By solving... (in , The polynomial coefficients are obtained from the set of pixels within a local window, which in turn generate the original spectral feature map. Each channel corresponds to polynomial coefficients of different orders, which can describe the distribution characteristics of spectral reflectance in more detail.

[0099] (2) Space purification treatment: The optimized vegetation active mask (Values ​​range from 0 to 1, where 1 represents vegetated areas and 0 represents non-vegetated areas) This serves as a spatial weight matrix for the original texture feature map. and original spectral feature map Perform pixel-by-pixel mask weighting. The specific formula is: ,in This represents element-wise multiplication. Through this operation, features from non-vegetated areas (mask value 0) are suppressed, while features from vegetated areas (mask value 1) are preserved, resulting in a spatially purified texture feature map. and spectral feature map This effectively reduces noise interference in non-vegetated areas, allowing subsequent treatments to focus more on the characteristics of green plants.

[0100] (3) Channel saliency weight calculation: The texture feature map after spatial purification is calculated separately through a dual-channel attention mechanism. and spectral feature map Channel saliency weights. For texture feature maps. First, perform a global average pooling operation on it to obtain... , Then, feature transformation is performed through a two-layer fully connected network:

[0101] ;

[0102] in For activation functions (such as the Sigmoid function). For the weight matrix ( The dimension after dimensionality reduction can be set according to the actual situation, such as... ), This is the bias vector. The final result is the texture feature channel weight vector. .

[0103] Similarly, for spectral feature maps After similar global average pooling and two fully connected network operations, the spectral feature channel weight vector is obtained. ,in , For the weight matrix ( The dimension after dimensionality reduction can be set according to the actual situation, such as... ) , This is the bias vector. This dual-channel attention mechanism enhances the focus on key feature channels by learning the importance of different channels.

[0104] (4) Cross-modal channel fusion: based on texture feature channel weight vector and spectral feature channel weight vector Texture feature map after space purification and spectral feature map Perform cross-modal channel fusion. The specific formula is: ,in This represents a weighted operation along the channel dimension, where corresponding channels are multiplied and then summed. This fusion operation organically combines texture and spectral features at the channel level, generating a spatiotemporally correlated feature tensor. ( (Based on the size changes after the fusion operation), it provides richer and more representative feature information for subsequent monitoring of green plant flowering periods.

[0105] As one example, the step of mapping the climate parameters currently collected by the environmental sensor on the target green plant into an environmental feature vector, and then interacting the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross-attention mechanism to generate a spatiotemporal decision feature vector for environmental perception includes the following sub-steps:

[0106] The climate parameters currently collected by the environmental sensors installed on the target green plant are feature-encoded to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters;

[0107] The spatiotemporal correlation feature tensor is transformed linearly to generate a cross-modal query vector, and the multidimensional environmental feature vector is mapped to a key vector and a value vector respectively.

[0108] Calculate the similarity matrix between the cross-modal query vector and the key vector to generate environment-aware attention weights;

[0109] The value vector is weighted and aggregated according to the attention weights to output the spatiotemporal decision feature vector.

[0110] In this embodiment, environmental sensors placed at the target greenery first collect climate parameters such as temperature, humidity, and light intensity, and encode these parameters to obtain a multi-dimensional environmental feature vector to quantify environmental information. Next, a linear transformation is performed on the previously generated spatiotemporal correlation feature tensor to obtain a cross-modal query vector. Simultaneously, the multi-dimensional environmental feature vector is mapped to key and value vectors to prepare for subsequent interactions. Then, the similarity matrix between the cross-modal query vector and the key vector is calculated to generate attention weights for environmental perception, clarifying the correlation between environmental factors and image features. Finally, the value vectors are weighted and aggregated based on these attention weights to generate a spatiotemporal decision feature vector for environmental perception. In summary, this embodiment breaks the isolation between image features and environmental factors, achieving effective interaction between the two through a cross-modal cross-attention mechanism. It fully considers the impact of environmental factors on the flowering period of greenery, enabling the final generated feature vector to comprehensively reflect the growth status and environment of the greenery, improving the comprehensiveness and accuracy of greenery flowering period monitoring, and providing more reliable feature support for subsequent flowering stage judgment.

[0111] To facilitate understanding of this embodiment, the following detailed description is provided:

[0112] First, environmental sensors collect the temperature of the target green plants in real time. ,humidity and light intensity Parameters. Since these three parameters have different dimensions and numerical ranges, they need to be standardized first. A standardization method based on dynamic thresholds is adopted, whereby the dynamic thresholds are calculated from the quantiles of historical data.

[0113] Set a dynamic lower threshold for temperature. and upper limit threshold These are the 5th and 95th percentiles of historical temperature data, respectively, obtained by analyzing hourly temperature data collected over the past year. The results are determined after sorting. The thresholds for humidity and light intensity are calculated similarly, yielding the following results: and The standardization process is as follows:

[0114] ;

[0115] in, As an adaptive adjustment coefficient, it is dynamically adjusted according to seasonal changes. For example, in summer, the weight of temperature's influence on flowering period increases, so it can be appropriately increased. The value was determined through correlation analysis between historical flowering data and climate parameters.

[0116] Next, in order to extract higher-order relationships between climate parameters, a multidimensional environmental feature vector is constructed using tensor product. :

[0117] ;

[0118] in, This represents the tensor product operation, which expands the one-dimensional standardized parameters into a nine-dimensional feature vector, fully exploring the interaction information between the parameters.

[0119] Then perform vector mapping: Let the spatiotemporal correlation feature tensor obtained above be... ,in For tensor height, For width, This represents the number of channels.

[0120] Generating cross-modal query vectors: on spatiotemporal correlated feature tensors Perform a pyramid pooling operation to divide it into three sub-regions of different scales, namely... , Average pooling is performed on each sub-region to obtain feature vectors of different scales. .

[0121] After concatenating these feature vectors, dimensionality reduction is performed using a two-layer fully connected network.

[0122] ;

[0123] in, For Swish activation functions, This is the weight matrix. For bias vectors, For intermediate layer dimensions, To query the vector dimension, these parameters are all updated via backpropagation during model training.

[0124] Generate key vectors and value vectors: transform multidimensional environmental feature vectors Key vectors are generated through different linear transformations. Sum value vector : , ,in, This is the weight matrix. For bias vectors, and The dimensions of the key vector and value vector are determined through hyperparameter optimization.

[0125] Next, the similarity matrix and the generated attention weights are calculated: a modified cosine similarity combined with Local Sensitive Hashing (LSH) is used to compute the cross-modal query vector. With key vector The similarity.

[0126] Will and Using locality-sensitive hash functions respectively and Mapping to hash buckets reduces computation: ,in, For random projection vectors, This is the offset. These parameters, representing the hash bucket width, were determined through multiple experiments and optimizations.

[0127] For matching vector pairs in the hash bucket, calculate the improved cosine similarity: ,in, To prevent the minimum value where the denominator is zero, The weighting coefficients are dynamically adjusted based on the plant species and growth stage, and are predicted by a classification model trained on historical data.

[0128] Similarity matrix After processing with the Softmax function, an environment-aware attention weight matrix is ​​generated. : .

[0129] Finally, weighted aggregation generates spatiotemporal decision feature vectors: based on the attention weight matrix. value vector Perform weighted aggregation: ,in, This is the final spatiotemporal decision feature vector for environmental perception, which integrates information from spatiotemporal correlation features and environmental features, providing a key basis for subsequent judgment of the flowering stage of green plants.

[0130] As one example, the step of inputting the spatiotemporal decision feature vector into a pre-constructed cascaded residual classifier, and generating the probability distribution of the flowering stage of the target green plant through the cross-layer residual connections of the cascaded residual classifier, includes the following sub-steps:

[0131] The spatiotemporal decision feature vector is input into the first residual module of the pre-constructed cascaded residual classifier. Local features are extracted through the convolutional layer, and the extraction results are residually connected with the original input features to generate the first-level enhanced features.

[0132] The first-level enhanced features are input into the second residual module in the cascaded residual classifier. The weight coefficients of each feature channel are generated through the channel attention mechanism, and dynamic pooling is performed on the feature map to generate the second-level enhanced features.

[0133] The second-level enhanced features are mapped and normalized using a fully connected layer to output the probability distribution of the flowering stage.

[0134] In this embodiment, firstly, the spatiotemporal decision feature vector is input into the first residual module of the cascaded residual classifier. Local subtle features are extracted using convolutional layers, and the extracted results are combined with the original input features through residual connections. This avoids gradient vanishing while enhancing the expressive power of the features, generating a first-level enhanced feature. Next, the first-level enhanced feature is fed into the second residual module. A channel attention mechanism is used to calculate the weight coefficients of each feature channel, highlighting key channel information. Dynamic pooling further filters effective features, generating a second-level enhanced feature. Finally, the second-level enhanced feature is mapped using a fully connected layer, converting it into a dimension corresponding to the number of flowering stages. After probability normalization, the probability distribution of the target plant's flowering stage is output. In summary, this embodiment, through the cascaded residual structure and the collaborative operation of multiple mechanisms, effectively enhances feature extraction and fusion capabilities, overcomes the training difficulties of deep networks, and significantly improves the accuracy and stability of classifying the flowering stages of plants.

[0135] The specific working process of this embodiment is as follows:

[0136] Step 1, First Residual Module Processing – Local Feature Extraction and Residual Connection: Let the spatiotemporal decision feature vector output from the above embodiment be... ,in The feature dimension is determined by the output dimension of the cross-modal attention mechanism. The first residual module contains deformable convolutional layers and adaptive residual connections, specifically implemented as follows:

[0137] 1. Feature extraction from deformable convolutional layers: Deformable convolutional kernels are used. ,in The number of output channels (dynamically set according to the number of flowering stages, such as...) , (Total number of flowering stages) The spatial size of the convolution kernel (e.g.) (This represents a 3×3 deformable convolution kernel). Deformable convolution learns offsets. The formula for offsetting traditional grid sampling points is: ,in, The current pixel coordinates, The coordinates of the sampling points of the traditional convolution kernel. For learnable offsets, This is the bias vector. This operation enhances the ability to capture irregular green plant texture features by dynamically adjusting the sampling position.

[0138] 2. Nonlinear activation and residual connection: for convolution output Applying an improved Swish-B activation function: ,in These are channel-level learnable parameters. By adaptively adjusting the shape of the activation function curve, the nonlinear feature representation is optimized.

[0139] Due to input With convolution output The dimensions may be inconsistent (e.g.) ), using weighted residual connections: ,in The residual connection weight matrix is ​​learned synchronously with the main network parameters through backpropagation to ensure effective fusion of cross-layer information.

[0140] Step 2, Second Residual Module Processing – Channel Attention and Dynamic Pooling: This process processes the first-level enhanced features... The input is a second residual module, which further enhances feature discriminativeness through a dual-channel attention mechanism and adaptive dynamic pooling:

[0141] 1. Channel attention weight generation: First, for... Perform global mean-variance pooling to generate channel-level statistics: ,in For channel indexing, The feature map space size (if the input is a vector) ).

[0142] Then, channel weights are generated using a multilayer perceptron (MLP) and gating mechanism. :

[0143] ,in , This is the weight matrix. To reduce dimensionality, The bias vector is learned to automatically suppress irrelevant channel noise.

[0144] 2. Channel weighting and dynamic pooling: for Perform channel weighting: ,in This indicates element-wise multiplication along the channel dimension. Subsequently, adaptive threshold dynamic pooling is employed, dynamically adjusting the pooling region based on the feature distribution. ,in These are adaptive coefficients (determined through training with historical flowering data). For feature values ​​greater than the threshold The set of pixels, This represents the number of elements in the set. This operation focuses on highly discriminative regions, avoiding information loss common in traditional pooling.

[0145] Step 3, Fully Connected Layer Mapping and Probability Normalization: Mapping the second-level enhanced features The input classification header contains cascaded fully connected layers and temperature-scaling Softmax. The specific steps are as follows:

[0146] 1. Multi-layer fully connected mapping: First, dimensional transformation is performed through the first fully connected layer:

[0147] ,in For the weight matrix ( (for intermediate layer dimensions) This is the bias vector.

[0148] Then, an unnormalized score is generated through a second fully connected layer: ,in This is the weight matrix. For bias vectors, This represents the total number of flowering stages (such as bud stage, bud stage, flowering stage, and withering stage).

[0149] 2. Temperature scaling probability normalization: Introducing temperature parameters Adjust the sharpness of the Softmax distribution: , ,in By dynamically optimizing the cross-entropy loss on the validation set, we ensure that the probability distribution conforms to the uncertainty characteristics of the actual flowering stage.

[0150] In summary, this embodiment addresses the shortcomings of traditional convolutional methods in extracting features from irregular plant morphologies by learning sampling offsets and cross-layer weights. Residual connections effectively mitigate gradient vanishing in deep networks. By combining mean-variance pooling and gating mechanisms to generate channel weights, dynamic pooling adaptively selects effective regions based on feature distribution, enhancing focus on environmentally sensitive features. Furthermore, adjustable temperature parameters optimize the probability distribution to adapt to differences in inter-class similarity across different plant varieties' flowering stages, improving classification robustness.

[0151] As one example, the step of generating the flowering period monitoring results of the target green plant by fusing the probability distribution of the flowering period stage with historical flowering period data based on a one-way temporal gating network includes the following sub-steps:

[0152] The probability distribution of the flowering stage is concatenated with historical flowering data in chronological order to form a time-series input sequence;

[0153] The time-series input sequence is modeled by a one-way gated loop unit, the forgetting ratio of historical states is controlled by a reset gate, and the fusion weight of the current input is adjusted by an update gate to generate time-series fusion features.

[0154] The time-series fusion features are used to make classification decisions, and the flowering period monitoring results are output.

[0155] In this embodiment, firstly, the current flowering stage probability distribution and historical flowering data are concatenated chronologically to construct a time-series input sequence, integrating flowering information from different time points to provide a data foundation for analyzing flowering trend changes. Then, a one-way gated loop unit is used to model the time-series input sequence. A reset gate flexibly controls the forgetting ratio of historical states to avoid interference from invalid historical information. Simultaneously, an update gate adjusts the fusion weights of the current input, adaptively fusing new and historical information to generate a time-series fusion feature containing time-dimensional characteristics. Finally, classification decisions are made on the time-series fusion feature to output the final flowering monitoring result. In summary, this embodiment, through in-depth mining and intelligent processing of time-series data, fully utilizes the patterns and trends contained in historical flowering data, combined with the current flowering probability distribution, effectively improving the accuracy and foresight of flowering monitoring, enabling more timely and accurate prediction of changes in the flowering stage of green plants.

[0156] To facilitate understanding of this embodiment, the following detailed description is provided:

[0157] Step 1, Constructing the Time-Series Input Sequence: Let the probability distribution of the current target plant's flowering stage be... ,in This represents the total number of flowering stages (e.g., bud stage, budding stage, flowering stage, and withering stage). Historical flowering data includes past periods. A probability distribution sequence with time steps Each .

[0158] To integrate current and historical data, the historical data is first time-aligned and its features enhanced. Since data collected at different times may have different sampling frequencies, an improved version of the Dynamic Time Warping (DTW) algorithm is used for alignment, as shown in the following formula:

[0159] ,in Indicates the first in historical data The and the first Alignment distance of probability distributions at each time step The distance is Euclidean. The optimal alignment sequence is found by backtracking the path. .

[0160] Then, trend features are extracted from the aligned historical data. The rate of change of the probability distribution between adjacent time steps is calculated: ,in The time interval between adjacent time steps is represented by [value]. The rate of change feature is concatenated with the original probability distribution to obtain the enhanced historical data. .

[0161] Finally, the current probability distribution Compared with enhanced historical data Concatenate the sequences in chronological order to generate a time-series input sequence. : .

[0162] Step 2, Modeling a Unidirectional Gated Recurrent Unit (UGRU): This involves modeling the timing input sequence... The input is fed into a custom one-way gated recurrent unit (UGRU). The UGRU achieves selective forgetting of historical states and adaptive fusion of the current input through a double reset gate and a dynamic update gate mechanism. The specific formula is as follows:

[0163] 1. Double reset door calculation: First reset door Used to control the forgetting of long-term historical states:

[0164] ;

[0165] Second reset door Used to control the forgetting of short-term historical states:

[0166] ;

[0167] in It is the Sigmoid activation function. It is the input weight matrix. It is the hidden state weight matrix. It is a bias vector. The dimension of the hidden layer is determined through hyperparameter optimization.

[0168] 2. Dynamic Update Gate Calculation: Considering the difference in the importance of data at different flowering stages, an attention-enhanced update gate is introduced. :

[0169] ;

[0170] in This is the weight matrix. For bias vectors, For attention weight vectors, This indicates element-wise multiplication.

[0171] 3. Hidden State Update: Calculate the current hidden state by combining the double reset gate and the dynamically updated gate. :

[0172] ;

[0173] in This is the weight matrix. This is the bias vector. After... After 10 time steps, the final hidden state This is the temporal fusion feature.

[0174] Step 3, Classification Decision and Result Output: Integrating Temporal Features The data is input into a multilayer perceptron (MLP) classifier to predict flowering period monitoring results. The MLP contains two fully connected layers:

[0175] 1. First fully connected layer:

[0176] ;

[0177] in This is the weight matrix. For bias vectors, The intermediate layer dimension is determined through hyperparameter optimization, and $\text{ReLU}$ is the modified linear unit activation function.

[0178] 2. Second fully connected layer:

[0179] ;

[0180] in This is the weight matrix. This is the bias vector.

[0181] For the second layer output Fuzzy probability normalization was performed to obtain the final probability distribution of the flowering period monitoring results. :

[0182] ;

[0183] in This is a correction coefficient calculated based on historical data fluctuations. It is used to adjust the confidence level of the probability distribution to ensure that the output results better reflect the actual flowering period variation patterns. The flowering period stage corresponding to the highest probability is the final flowering period monitoring result.

[0184] In summary, this embodiment addresses the issue of historical data temporal inconsistency through an improved DTW algorithm and enhances the dynamism of time-series information by utilizing rate-of-change features. Furthermore, UGRU achieves hierarchical forgetting of historical states through a dual-reset gate and dynamically adjusts the update gate weights using an attention mechanism, effectively fusing flowering period data across multiple time scales. Additionally, the introduction of a probability calculation method for correction coefficients enhances the adaptability of the results to data fluctuations and improves the reliability of the monitoring results.

[0185] See Figure 2 This is a schematic diagram of a deep learning-based green plant flowering period monitoring device according to an embodiment of the present invention. The deep learning-based green plant flowering period monitoring device includes:

[0186] Image generation module 10 is used to acquire the current time-series image set of the target green plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the time-series image set, and generate a standardized image set containing visible light image and near-infrared band image; wherein, the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolutional kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel.

[0187] Feature extraction module 11 is used to extract spectral reflectance features and texture features from the standardized image set, combine the vegetation activity mask and perform region-constrained fusion of the spectral reflectance features and texture features through a channel attention mechanism to generate a spatiotemporal correlation feature tensor; wherein, the region-constrained fusion suppresses noise interference in the non-vegetation areas of the target green plant through mask weighting.

[0188] The vector generation module 12 is used to map the climate parameters currently collected by the environmental sensor set on the target green plant into an environmental feature vector, and to interact the environmental feature vector with the spatiotemporal correlation feature tensor through a cross-modal cross attention mechanism to generate a spatiotemporal decision feature vector for environmental perception.

[0189] The probability prediction module 13 is used to input the spatiotemporal decision feature vector into a pre-constructed cascaded residual classifier, and generate the flowering stage probability distribution of the target green plant through the cross-layer residual connection of the cascaded residual classifier.

[0190] Flowering period analysis module 14 is used to generate the flowering period monitoring results of the target green plant by fusing the probability distribution of the flowering period stage with historical flowering period data based on a one-way temporal gating network.

[0191] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0192] This invention acquires a time-series image set of target green plants through an image acquisition device. After cross-modal dynamic enhancement processing, a learnable convolutional kernel is used to separate the brightness and color components. A vegetation activity mask is generated based on the spectral reflectance of the near-infrared channel, resulting in a standardized image set containing visible and near-infrared band images, thereby quantifying the physiological activity state of the vegetation. Then, the spectral reflectance and texture features of the standardized image set are extracted and combined with the vegetation activity mask. A channel attention mechanism is used for region-constrained fusion to suppress noise interference in non-vegetation areas, generating a spatiotemporal correlation feature tensor. Climate parameters collected by environmental sensors are mapped into environmental feature vectors. Through cross-modal cross-attention mechanism and interaction with the spatiotemporal correlation feature tensor, a spatiotemporal decision feature vector for environmental perception is generated, thereby dynamically monitoring the flowering period in conjunction with environmental parameters. Finally, the spatiotemporal decision feature vector is input into a cascaded residual classifier. Cross-layer residual connections are used to generate a probability distribution of the flowering period stage. Then, historical flowering period data is fused through a unidirectional temporal gating network to obtain the flowering period monitoring results. In summary, the embodiments of the present invention significantly improve the accuracy and environmental robustness of monitoring the flowering period of green plants in complex environments through a collaborative mechanism of cross-modal dynamic enhancement, regional constraint fusion, and environmental perception decision-making.

[0193] As one example, the image generation module is specifically used for:

[0194] Spatial registration processing is performed on the visible light band map and near-infrared band map of the time-series image set to generate a pixel-aligned dual-channel image set;

[0195] The dual-channel image set is input into a separable convolutional network. The brightness component features are extracted through the first convolutional branch, and the color component features are extracted through the second convolutional branch to generate an intermediate feature map that decouples brightness and color.

[0196] An initial vegetation activity mask is generated based on the spectral reflectance difference of the near-infrared band image, and a morphological closing operation is performed on the initial vegetation activity mask to generate a vegetation activity mask with continuous edges.

[0197] The intermediate feature map, which decouples brightness and color, is fused with the vegetation activity mask using channel weighting to output the standardized image set.

[0198] As one example, the feature extraction module is specifically used for:

[0199] Orientation gradient distribution analysis is performed on the visible light images in the standardized image set to extract the original texture feature map. At the same time, spectral reflectance distribution modeling is performed on the near-infrared band images in the standardized image set to generate the original spectral feature map.

[0200] Using the vegetation activity mask as a spatial weight matrix, pixel-by-pixel mask weighting operations are performed on the original texture feature map and the original spectral feature map to suppress feature noise in non-vegetation areas, resulting in spatially purified texture feature map and spectral feature map.

[0201] The channel saliency weights of the spatially purified texture feature map and spectral feature map are calculated separately using a dual-channel attention mechanism to generate texture feature channel weight vectors and spectral feature channel weight vectors.

[0202] Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and spectral feature map to generate the associated feature tensor.

[0203] It is understood that the above-mentioned embodiments of the deep learning-based green plant flowering period monitoring device can be referred to the above-mentioned embodiments of the deep learning-based green plant flowering period monitoring method, and will not be elaborated here.

[0204] See Figure 3 This is a schematic diagram of a deep learning-based green plant flowering period monitoring system according to an embodiment of the present invention. The deep learning-based green plant flowering period monitoring system of this embodiment includes: a processor 100, a memory 101, and a computer program stored in the memory 101 and executable on the processor 100, such as a deep learning-based green plant flowering period monitoring program. When the processor 100 executes the computer program, it implements the steps in the various deep learning-based green plant flowering period monitoring method embodiments described above. Alternatively, when the processor 100 executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0205] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the deep learning-based green plant flowering period monitoring system.

[0206] The deep learning-based green plant flowering period monitoring system can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The system may include, but is not limited to, processors and memory. Those skilled in the art will understand that the schematic diagram is merely an example of a deep learning-based green plant flowering period monitoring system and does not constitute a limitation on the system. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the deep learning-based green plant flowering period monitoring system may also include input / output devices, network access devices, buses, etc.

[0207] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the deep learning-based green plant flowering period monitoring system, connecting all parts of the system via various interfaces and lines.

[0208] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the deep learning-based green plant flowering period monitoring system by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0209] The modules / units integrated in the deep learning-based green plant flowering period monitoring system, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0210] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0211] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A deep learning-based green plant flowering monitoring method, characterized by, The method comprises the following steps: acquiring a current time series image set of a target green plant through an image acquisition device, performing cross-modal dynamic enhancement processing on the time series image set to generate a standardized image set containing visible light images and near-infrared band images; wherein the dynamic enhancement processing separates the brightness and color components of the target green plant through a learnable convolution kernel, and generates a vegetation activity mask of the target green plant based on the spectral reflectance of the near-infrared channel; extracting spectral reflectance features and texture features in the standardized image set, combining the vegetation activity mask and performing regional constraint fusion on the spectral reflectance features and texture features through a channel attention mechanism to generate a spatio-temporal correlation feature tensor; wherein the regional constraint fusion suppresses the noise interference of the non-vegetation region of the target green plant through mask weighting; mapping the climate parameters currently collected by the environmental sensor arranged in the target green plant into an environmental feature vector, and interacting the environmental feature vector with the spatio-temporal correlation feature tensor through a cross-modal cross-attention mechanism to generate an environment-aware spatio-temporal decision feature vector; inputting the spatio-temporal decision feature vector into a pre-constructed cascaded residual classifier, and generating a flowering stage probability distribution of the target green plant through the cross-layer residual connection of the cascaded residual classifier; fusing the flowering stage probability distribution and historical flowering data based on a one-way time series gating network to generate a flowering monitoring result of the target green plant. 2.The deep learning-based green plant flowering stage monitoring method of claim 1, wherein, The method comprises the following steps: performing spatial registration processing on the visible light band images and the near-infrared band images of the time series image set to generate a pixel-aligned dual-channel image set; inputting the dual-channel image set into a separable convolutional network, extracting brightness component features through a first convolution branch, extracting color component features through a second convolution branch, and generating intermediate feature maps with decoupled brightness and color; generating an initial vegetation activity mask based on the spectral reflectance difference of the near-infrared band images, and performing a morphological closing operation on the initial vegetation activity mask to generate an edge-continuous vegetation activity mask; performing channel weighting fusion on the intermediate feature maps with decoupled brightness and color and the vegetation activity mask to output the standardized image set. 3.The deep learning-based green plant flowering stage monitoring method of claim 2, wherein, The method comprises the following steps: performing direction gradient distribution analysis on the visible light images in the standardized image set to extract an original texture feature map, and performing spectral reflectance distribution modeling on the near-infrared band images in the standardized image set to generate an original spectral feature map; The vegetation activity mask is taken as a spatial weight matrix, and pixel-by-pixel mask weighting operations are performed on the original texture feature map and the original spectral feature map respectively, so as to suppress feature noise in non-vegetation areas and obtain a spatially purified texture feature map and a spectral feature map; Channel significance weights of the spatially purified texture feature map and the spectral feature map are calculated through a double-channel attention mechanism, to generate a texture feature channel weight vector and a spectral feature channel weight vector; Based on the texture feature channel weight vector and the spectral feature channel weight vector, cross-modal channel fusion is performed on the spatially purified texture feature map and the spectral feature map, to generate the correlation feature tensor. 4.The deep learning-based green plant flowering stage monitoring method of claim 3, wherein, The current climate parameters collected by the environmental sensor arranged in the target green plant are mapped into an environmental feature vector, and the environmental feature vector and the spatio-temporal correlation feature tensor are interacted through a cross-modal cross-attention mechanism, to generate an environment-aware spatio-temporal decision feature vector, including the following sub-steps: The current climate parameters collected by the environmental sensor arranged in the target green plant are feature-encoded to generate a multi-dimensional environmental feature vector; the climate parameters include temperature, humidity and light intensity parameters; The spatio-temporal correlation feature tensor is linearly transformed into a cross-modal query vector, and the multi-dimensional environmental feature vector is mapped into a key vector and a value vector respectively; A similarity matrix of the cross-modal query vector and the key vector is calculated to generate an environment-aware attention weight; The value vector is weighted and aggregated according to the attention weight, and the spatio-temporal decision feature vector is output. 5.The deep learning-based green plant flowering stage monitoring method of claim 4, wherein, The spatio-temporal decision feature vector is input into a pre-constructed cascaded residual classifier, and a flower stage probability distribution of the target green plant is generated through cross-layer residual connection of the cascaded residual classifier, including the following sub-steps: The spatio-temporal decision feature vector is input into a first residual module in the pre-constructed cascaded residual classifier, local features are extracted through a convolution layer, and the extracted results are residual-connected with original input features to generate first-level enhanced features; The first-level enhanced features are input into a second residual module in the cascaded residual classifier, weight coefficients of each feature channel are generated through a channel attention mechanism, and dynamic pooling processing is performed on the feature map to generate second-level enhanced features; The second-level enhanced features are mapped through a full connection layer and are subjected to probability normalization processing, and the flower stage probability distribution is output. 6.The deep learning-based green plant flowering stage monitoring method of claim 5, wherein, The flower stage probability distribution and historical flower stage data are fused based on a one-way time sequence gating network to generate a flower stage monitoring result of the target green plant, including the following sub-steps: The flower stage probability distribution and the historical flower stage data are spliced into a time sequence input sequence in chronological order; The time sequence input sequence is modeled through a one-way gated recurrent unit, a forgetting ratio of a historical state is controlled through a reset gate, and a fusion weight of a current input is adjusted through an update gate, to generate a time sequence fusion feature; The time sequence fusion feature is subjected to classification decision, and the flower stage monitoring result is output. 7.A deep learning-based green plant flowering monitoring device, characterized by The method comprises the following steps: The image generation module is configured to acquire a current time-series image set of the target green plant through an image acquisition device, perform cross-modal dynamic enhancement processing on the time-series image set, and generate a standardized image set containing a visible light image and a near-infrared waveband image; wherein the dynamic enhancement processing separates luminance and color components of the target green plant through a learnable convolution kernel, and generates a vegetation activity mask of the target green plant based on spectral reflectance of a near-infrared channel; The feature extraction module is configured to extract spectral reflectance features and texture features in the standardized image set, combine the vegetation activity mask, and perform regional constraint fusion on the spectral reflectance features and the texture features through a channel attention mechanism, to generate a spatio-temporal correlation feature tensor; wherein the regional constraint fusion suppresses noise interference of non-vegetation regions of the target green plant through mask weighting. The vector generation module is configured to map climate parameters currently collected by an environmental sensor arranged in the target green plant into an environmental feature vector, interact the environmental feature vector with the spatio-temporal correlation feature tensor through a cross-modal cross-attention mechanism, and generate an environment-aware spatio-temporal decision feature vector. The probability prediction module is configured to input the spatio-temporal decision feature vector into a pre-constructed cascaded residual classifier, and generate a flowering stage probability distribution of the target green plant through cross-layer residual connection of the cascaded residual classifier. The flowering analysis module is configured to fuse the flowering stage probability distribution and historical flowering data based on a one-way time-series gating network, and generate a flowering monitoring result of the target green plant.

8. The deep learning-based greenery flowering stage monitoring apparatus of claim 7, wherein, The image generation module is specifically configured to: perform spatial registration processing on visible light waveband images and near-infrared waveband images of the time-series image set, to generate a pixel-aligned dual-channel image set; input the dual-channel image set into a separable convolutional network, extract luminance component features through a first convolution branch, extract color component features through a second convolution branch, and generate an intermediate feature map with luminance and color decoupled; generate an initial vegetation activity mask based on spectral reflectance differences of the near-infrared waveband images, and perform a morphological closing operation on the initial vegetation activity mask, to generate a vegetation activity mask with continuous edges; perform channel weighting fusion on the intermediate feature map with luminance and color decoupled and the vegetation activity mask, and output the standardized image set.

9. The deep learning-based greenery flowering stage monitoring apparatus of claim 8, wherein, The feature extraction module is specifically configured to: perform directional gradient distribution analysis on visible light images in the standardized image set, to extract an original texture feature map, and perform spectral reflectance distribution modeling on near-infrared waveband images in the standardized image set, to generate an original spectral feature map; use the vegetation activity mask as a spatial weight matrix, and perform pixel-by-pixel mask weighting operation on the original texture feature map and the original spectral feature map respectively, to suppress feature noise in non-vegetation regions, to obtain a spatially purified texture feature map and a spectral feature map; calculate channel saliency weights of the spatially purified texture feature map and the spectral feature map respectively through a dual-channel attention mechanism, to generate a texture feature channel weight vector and a spectral feature channel weight vector. Based on the texture feature channel weight vector and the spectral feature channel weight vector, the spatially purified texture feature map and the spectral feature map are cross-modal channel fused to generate the correlation feature tensor. 10.A deep learning-based green plant flowering monitoring system, characterized in that, The application discloses a deep learning-based green plant flowering period monitoring method, and relates to the technical field of computer vision. The application discloses a deep learning-based green plant flowering period monitoring method, and relates to the technical field of computer vision.

Images